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Austin, Texas, was designed "-o answer some very' specif ic questions 
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socioeconomic status elementary schools that results in greater 
s»-udent time 01 task, exposure to content, and achievement. The 




corstrain*s imposed or.^^-eachers. This report details the history of 
*he study, the training course received by observers, and data 
collection activities, and summarizes preliminary findings from the 
ptudy. More effective organizers appeared to: (1| have thought in 
advance about rules and procedures necessary, and to have established 
them before problems arose: (2) be able to plan activities and 



procedures with a student's perspective: and (3) introduce 
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THE CLASSROOM ORGANIZATION STUDY: 
Methodology and Instrumentation \ 

The Classroom Organization Study (COS), was part of the Austin 
Independent School District Project (AISD Project), which involved 
collaboration and cooperation between three agencies: 

1. The Research and Development Center for Teacher Education at 
The University of Texas at Austin (R&D Center); 

2. The Austin Independent School District (AISD) ; and 

3. The National Institute of Education. 

During 1977-78, two major efforts were carried Dut through the AISD 
Project: the Classroom Organization Study, and assj.sting the District 
in the development of a Teacher Evaluation instrument. This study was 
designed to answer questions about effective instruction in low SES 
elementary schools by focusing on classroom organ i.^, at ion and management, 
particularly those steps thst are important at the beginning of the 
school year. This report details the history of the study, describes 
data collection activities, and summarizes preliminary findings .from the 
study. 



TIhE LINE FOR 



School year, 1976-77 



Spring, 1977 



Summer, . 1977 



THE CLASSROOM ORGANIZATION STUDY 

Production by COET staff for AISD of a series 
of research reviews on effective teaching 
in low SES elementary schools. 

\ 

Meetinl^s with AISD administrators in the 
• Division of Instruction to discuss 
possible research of mutual benefit. 



August 22-26, 1977 



Planning for a study to be done in AISD 

.^/focusing on organization and management in 
third-grade classes with large proportions 
of students , from low SES backgrounds. 
Meetings were held with principals from 10 
elementary schools regarding the study. 

MeeTirTgs with faculties of eight schools to 
di:;^uss the study and obtain teacher 
participation. 

Observer training for 19 observers at the R&D 
Center. 



August 29, 1977 



August 29, 1977- 

September 16, 1977 



September 19-2^, 1977 



October 6-26, 1977 



October 3 , 1977- 

November 22, 1977 

November 7, 1977- 
May 30, 1978 



February 13-17, 1978 



May 19, 1978- 
June 28, 1978 



First day of school. Observations in 13 
third-grade c lass rooms . 

First three weeks of school. Twenty-eight 
teachers in eight schools were seen by 19 
observers for 243 observations of 
approximately 2.5 hours in length. 

Observer debriefing. Observers filled out 
summary ratings and answered ^questions 
about the teachers they had observed. 



First interviews of teachers, to collect 
information about beginning of school 
organization. 



CBAM interviews of teachers. 

Twenty-seven teachers observed by four 
observers for a tot^al of 257 
observations. 

Reliability observations . Groups of two 
observers in the same classrooms. 



End of school interviews of teachers. 
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BACKGROUND OF THE STUDY 

i> 

Much work done at the R&D Center over the last ' few years by ' the 
Correlates of Effective Teaching Program (COET) has focused on classroom 
pr.ocnsses which related to achievement in the basic skill areas in 
. elementary schools. This interest and general research background led 
to one . component of the first year of the AISD Project in which the COET 
staff produced a series of research reviews on effective teaching in low 
SES elementary schools. (Other Center programs were also involved in 
other components.) The reviews included research done in Austin Schools 
by the Center, as well as work by researchers in other parts of the 
country. They covered the topics of in-service, classroom structure, 
teaching methods, teacher-student interaction, and the use of 
instructional time. • Each was produced in two forms: a 15-20 page paper, 
and a 5-6 page Version that summarized the highlights of the research. 
These reviews were disseminated in the AISD by the Center and by the 
AISD Office of Research and Evaluation (ORE) with whom the Center had 
worked closely in preparing the reviews. The papers were part of one of 
ORE'S priority efforts that year, which was_the gathering of information 
for district personnel about research findings on instruction of low SES 
students. ORE prepared summaries for other topics, and the R&D Center 
prepared the reviews of research on classroom processes. The summaries 
were very well-received throughout the District. 

One result of these summaries was that R&D Center- staff met with 
AISD administrators in the Division of 'Instruction and then began to 
discuss how the Center researchers might be of furthur use to 



practitioners in the p|.strict. These contacts resulted in a list of 
research questions about effective teaching in elementary schools, 
especially low SES schools, 'which were of high concern to the staff of 
the Division of Instruction* y 

One set of questions on this list was about classroom organization 
and management. Since much past R&D research had yielded conclusions 
that these were extremely important facets of teaching, especially in 
low SES schools, it was decided thato this topic could be most 
effectively researched by persons at the Center. Indeed, all of the 
research summaries prepared by the Center, and much original work done 
here, had expressed the importance of effective organization and '^^ 
management techniques which resulted in greater student time in academic 
tasks and greater involvement with and exposure to academic content. 

However, very little was known about what specific teacher 
behaviors result in "better" organizat ibn . In particular, little 
information was available about what factors are most important in 
establishing a smooth-running classroom at the beginning of the year. A 
wealth of general advice is available that ranges from "Don't smile 
until Christmas" to "Have your room organized and ready on the first day 
of school." However, it was felt that such general statements ware not 
.sufficient to help a teacher learn effective organization, especially 
when (s)he has never taught before. General principles of instruction 
are useful, but they must be illustrated and supported by concrete 
examples if they are to be internalized by new and inexperienced ^ 
teachers . 

Therefore, the Classroom Organization Study was designed to answer 
some very specific questions about establishing and maintaining 



classroom organization that results in greater student time on task . ' 
exposure to content ; and achievement . The ultimate purpose of the study 
was to produce knowledge .that could be expressed in terms of specific 
teacher behaviors that produce effective management of tiine, 
instructional materials , contacts between the teacher and students,, 
student participation in classroom activities, and the external 
constraints imposed on teachers. These topics also reflected specific 
district concerns. It was hoped that the study would also yield new 
ways of conceptualizing classroom organization. 

After several informal discussions with AISD staff, a proposal was 
sent to the Director of Elementary Education which presented objectives 
which would be addressed by the study. This proposal follows this 
discussion. ^ 

The response to this proposal was very positive, and it\was agreed 
to conduct a study during the school year 1977-1978, to focus on 
organization and management in third-grade classes in schools with large 
proportions of students from low SES . backgrounds . There fore , during the 
summer of 1977, planning for the study was begun. Three program areas 
in the Center were included in the Classroom Organization Study during 
the planning period: 

1. " The Correlates of Effective Teaching Program (COET) was 
responsible for designing the instruments for observation, training 
observers, and setting up observations in the schools, as well as 
conducting two interviews of each participating t;^acher. 

2. The Procedures for Adopting Educational Innovations /Concerns- 
Based Adoption Model (PAEI/CBAM) Program was responsible for 
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''interviewing the teachers in the sample using the focused interview 
techniques developed by them. 

3* The Evaluation of Teaching Program (EOT) was involved in some 
of the quantitative data analyses of the study. 

During the planning stage, COET staff members visited with 
principals and teachers in the District to solicit cooperation. 

Principals in 10 schools were contacted and the proposed study was 
discussed with them. Generally, the reaction of the principals was 
quite favorable, and they arranged fo.r a meeting with their third-grade 
teachers during the week before school* 

Ultimately, 29 teachers in eight schools agreed to participate in 
the study. In the initial meeting with the teachers, the researchers 
explained the purposes and procedures ojf the study. A teacher handout 
describing the study and the teachers' ro^le. follows . * 

During the first week of school, one teacher decided not to 
participate, and after the first three weells another tether was dropped 
from the sample because she went on maternity leave and did not return 
to school. Therefore, the sample was composed of 27 teachers. Two of 



the 27 teachers were transferred to other grade leveTs J second and 
fourth), but observations of them continued in their new classes. 

Throughout the data collection period, the researchers communicated 
with the teachers periodically to advise them of the progress of the 
study. On the wbiole, the rapport established with the teachers was 
excellent, and the observers were accepted in the, classroom with few 
problems . 
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Corrospondonco with Pnrticilpantjs 
Teacher Handouts 

. ■ . / 

Dear Teacher: j 

Since we hope that you and your third graders will be participating with 
us this year in- the A.I.S.D. Classroom Organization StUy, we would like to 
introduce ourselves and describe briefly what we hope to achieve in this project, 

We will be sitting in on a nujibe.. of third grade classrooms throughout 
the school year in order to observe different types of classroom organization. 
We are interested in seeing how Austin teachers organize their classrooms and 
manage instructional materials, instructional time, contacts with individual 
pupils, and activities. With the information we have gathered directly from 
classrooms we plan to formulate specific" suggestions and advice on classroom 
organization which will be of practical help to other teachers, school 
district personnel, and teacher educators. In other words, we would like to 
see the AISD Classroom Organization Study complete the full circle from 
classrooms like yours back to many other classrooms. 

We know that you have questions about the project and'that you may want 

more information on which to base, your decision to ■ participate in this study. 

Below are answers to a series of likely questions, and we will also provide 

answers to other questions in person. . 

What is the background of the "AISD Classroom Organization Study? 

The study is a cooperative venture between the School District and the 
R&D Center for Teacher Education. The District^as become increasingly, 
concerned with the amount of real "learning time" available to Austin class- 
rooms, and many individuals have become interested in how different teachers 
cope with outside constraints and organizational problems to provide the * 
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greatest amount of teaching and learning activity. These coneerns echo the 
Center's long-range mission, which involves the Center in "developing 
principles and alternative strategies which different t-achers can employ 
to maximize their. impact on child learning . , , and in identifying the 
contextual factors in school and community which can inhibit or be organized 
to support teaching and learning." 

Throughout our planning this summer we have been pleased that so many 
administrators, principals, coordinators, and teacher-consultants have expressed 
enthusiasm for such a study. Your principal was one of. those who agreed to let 
us observe in third grade classrooms if the teachers wished to participate. 
Since we at the Center have always felt very strongly about the necessity for 
teachers and researchers to work toget^ ar for the, improvement of the classroom 
process,, we are looking forward to wor;.ing with you in the AISD Classroom 
Organization Study. 

What is the focus of the AISD Classroom Organization Study? 

The study will look at various ways in which teachers organize their 
classrooms at the beginning of the year and how they maintain thair organiza- 
tion throughout the school year. Because of this focus we plan to undertake 
what we call "naturalistic^^ classroom observation," which means that we will 
sit in your classroom for certain periods as quietly and unobtrusively as we 
can. We will be observing very intensively during the first three weeks of 
school and then periodically through the rest of the year. 

For a number of reasons we have decided to conduct the study in schools 
which receive a significant amount of Title I funding or which are near 
qualifying for Title I funds. The study will involve about 35 third grade 
classroOTDs in Austin, We will concentrate on several areas of classroom 
organization: what are some defi^ige techniques teachers use in handling 
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instructional materials? how do they organize activities? and what are the 
internal and external constraints that interfere with a teacher's ability to 
create a "smoothly-running" classroom? 

The study will analyze the information obtained from your classrooms 
in terms of specific teaching and organizational techniques which are seen as 
desirable trends. We believe that this information will be helpful to new 
teachers in the AISD or teachers who desire some specific suggestions on ways 
of improving their classroom organization. 

What will I have to do if I participate in this study? 

First of all, we wish to emphasize that it is your decision whether or 
not to participate in the Classroom Organization Study. Your principal has 
merely given us permission to contact you about the project. Naturally , as 
with any study of this nature, your privacy will be carefully protected and 
no confidential or identifying information about' you or your classroom will be 
made availab4.e; 

There are three activities which we will be asking participating teachers 
to undertake: 

!• Observers allowed in the classroom 9 to 10 times during the first 
three^ weeks of school (probably for a half-day each time) and then once every 
3 to 4 weeks thereafter (for a day or two half-days within the same week) . 
As was mentioned previously, our observers have been trained to be as unobtrusive 
as possible in your classroom, and past research'* pro j ects have proved this 
training to be quite successful. Estimated time required: none. 

2. Participate in. three interviews during the year . We plan to interview 
participating teachers after the third week of school,, in January, and again in 
April. These will be short interviews focusing on your organizational techniques 
and your observations about your classroom. Estimated time requited: one hour each 
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3- Complete questionnaires. We plan to ask participating teachers to 
fai out short questionnaires twice during the study, once after the third 
week of school and again at the end of the year. Estimated time required: 
30 minutes each. 

What will I receiv e in return for participating in this study ? 

^' Honorarium for the e xtra time you spend during the year : We will pay 
-each participating teacher $100.00 for time >and effort; we hope to divide the 
sum for payments before Christmas and at tTie end of the school year. 

2- Personalized feedback; At the end of the year you will receive 
personalized feedback based on our intensive observation of your classroom, 
and we would be happy, if you wish, to provide comments on the organizational 
and management processes which we saw throughout the year. We plan to structure 
our analysis of your individual classroom in the larger perspective of teaching 
and organizational trends observed in other Austin classrooms. 

3- Written materials; You will receive copies of the results of the 
study as soon as they are available, along with other informal newsletters 
about the project. 



15 



1-12= 



I 



SECTION II 
DATA COLLECTION 



16 

id ■ ■ ^^-1 

ERIC 



SECTION II 
DATA COLLECTION 



There are several sources of information in the study. Each is 
described briefly, below. 

Data collection was divided into four areas: 

1. Classroom observations in the first three weeks 

2. Teacher interviews and questionnaires 

3. Classroom observations after November 

4. Summary observer ratings. 
Classroom Observations in the First Three Weeks 

Each of the 27 participating teachers comprising the final sample 
was seen eight or nine times during the first three weeks of school 
yielding over 600 hours of observation. Approximately one-half of the 
teachers were seen on the morning of the first day, and the rest were 
seen on the morning of the second day. Each teacher was observed during 
both mornings and afternoons, with a ri.tio of about two morning 
observations to every afternoon one. 

Observer training. Eighteen observers participated in this phase 
of data collection. Many of them were former te.ichers, some were 
graduate students already involved in R&D work, and others were R&D 
staff members. All observers underwent a week's training which 
emphasized the nature of third grade classrooms and the types of teacher 
and student behaviors which were important to note . Concepts and terms 
used in the study were defined and discussed (a copy" follows). 
Observers also received pract ice and feedback based on their 
observations of video tapes. 



17 

1 1-3 



The observation sessions in the first three weeks resulted in four 
types of information. Examples of the forms and instructions for their 
use follow this section. 

i- The narrative record . While in the classroom, the observers 
maintained an extensive written record of classroom behavior, 
particularly that which involved organization and management. This 
narrative provided a chronology of the classroom events seen by the 
observer. The details recorded in the narrative were guided by 61 
questions described below under "Follow-up questions." The narrative 
technique was used because of the large number of questions and 
variables under consideration, the need to preserve a coherent 
description of the sequence of btehaviors and events, and the desire to 
obtain detailed anecdotal accounts of useful management strategies. 

2. Follow-up questions to the narrativg record . After completing 
a written narrative, the observer ' then supplied additional descriptions 
using the 61 questions. Answers to these were to have been included in 
the narrative when applicable. When information pertinent to a question 
had not been included in the narrative, the observers added the - 
information on a special form, or else they indicated that a particular 
question was not applicable to that day's observation. 

3. Student engagement ratin?>s . Since the amount of student 
t imc--on-task was regarded as a "process" measure of management 
effectiveness, i observers took a count every 15 minutes of the number 
of students who could be classified, in each of five categories of 
engagement: ■ 

On-task academic . The student was working on an academic 
assignment (e.g., writing an assignment), 
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b. On-task procedural. The student was performing a procedure or 
routine which was not academic in nature, but which was expected and 
. desired by the teacher (e.g., lining up for a transition). 

Off-task sanctioned . The student was not performing an 
academic or procedural task, but was still engaged in acceptable 
behavior (e.g., going to the bathroom). 

Off-task unsanctioned. The student was behaving inappro- 
priately for the setting or activity. Inappropriate behavior was con- 
sidered to occur when classroom rules or procedures were violated, or 
when the student was not engaged in expected academic or procedural 
behavior . 

e. Can't tell . When the observer could not confidently classify a 
student as belonging to one of the above categories, he or she was 
counted here . 

These ratings were modified for observations conducted after 
November. 

4. Component ratings. To supplement the narrative records, and 
give additional information about organizational and management behav- 
iors, 32 ratings of selected instructional ahd managerial characteris- 
tics were made after each observation. These items were considered com- 
ponents of an assessment system of nine goal areas: ' 

1- Assessing the needs and academic mastery of pupils 

2. Designing lessons to achieve particular purposes 

3. Locating, constructing, and using materials 

4. Presenting information clearly 

5. Developing skills and processes of reasoning 
^- Developing positive attitudes 
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7. Managing pupil beh.Hvior 

8. Interacting effectively 

9. Demonstrating self knowledge. 

This assessment system was included in the Classroom Organization 
Si.udy for two reasons. First, the system provi'l^s high inference 
ratings in several domains important for management, and thus it has the 
po'rential for identifying global areas that differentiate various levels 
of capability in organizing and structuring classroom activities. 
Second, the use of the assessment system provides a way to relate this 
research study to pre-service teacher education, since an earlier 
version of the assessment system was developed by a committee of faculty 
from U.T. Austin, including one of the AISD Project staff. The system 
was used on a pilot basis in the elementary teacher education program at 
The University of Texas; research on this system will allow these 
pre-service criteria to be validated against teachers in the field. 

Therefore, by the end of the first three weeks, each teacher had 
been observed at least eight times for at least two hours each visit, 
and each observation resulted in a detailed narrative, follow-up 
questions, student engagement ratings, and component ratings. These 
data provide the basis for an intensive examination of the beginning of 
school in a large number of classrooms. 
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Other Data Collected From Observers After the First Three Week s 

Each observer saw from two to five teachers several times each, and 
each teacher was seen by at least two observers. In order to get more 
general information from the observers, each was personally interviewed 
by the principal investigators in "debriefing" sessions. The questions 
asked in these observer interviews follow. 
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TRAINING riATERIALS INTRODUCTION 



This outline represents an intellectual framework to guide 
your observation of organization in the third grade classroom. To 
avoid the time-consuming task of specifying every conceivable activity 
in the classroom, this Narrative Guide is intended to help you focus • 
upon the three major concerns of this study: How do teachers manage — 
1. materials, 2. environmental constraints, 3, behavioral contacts? 

This narrative guide is not exhaustive in terms of possible 

classroom incidents. It is intended only to serve as a guide to use'' 

in preparation for classroom observation. It is not meant to be used 

as a checklist or as a set of specific questions to be answered. 

Preferably, the coder should study this narrative -guide before entering 

the classroom. This procedure will permit the observer to attend to 

..J - • ■ .■ ■■ 

the classroom without" hindrance (i,e,, the need to be constantly 

referring to pieces of paper) while keeping the observer focused on 

the principles of this study, - 
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Concepts and Terms used in the Classroom Organization Study 

Several major sets of concepts are used throughout the materials given 
you and the discussion. They are ways of breaking down classroom events into 
categories which can be discussed, compared, analyzed, etc. Before getting 
into the fine details of writing a narrative and using the other measures, 
it would be wise to become familiar with these concepts. 

Subject matter: the major categories of curriculum which are taught. Here 

are some you can expect to observe in third grade: reading, language arts, 
math, spelling, social studies, science, health, art. music. P.E. 

content: a finer definition of a curriculum subject. Content refers to the 
specific learning involved, and. when obvious, whether' the learning is 
new or old. For example, reading content might be learning long vowel 
sounds or reading for comprehension, Math might include division with 
one-place numbers, multiplication of .fractions, or learning to measure with 
^ the metric system. Social studies might include review of the chapter in 
the textbook on the fdmily or learning the locations of the seven continents 
Activity: what the students actually are doing to learn the content. For 

the example listed above, the activities involved might be reading at the ' 
student^^s desk and answering questions on a ditto sheet, working math 
, problems at the board in front of the class, listening to a teacher 

explanation, or measuring the desks in the room with a meter stick. .There 
can be more than one activity at a time for a student. (For example, 
listening to a tape and, responding to a worlcsheet at certain points on the 
tape). . I : . 



Materials: all objects, equipment, and supplies used by teachers or 

students in learning activities. The only time the students are not . 
using some material is when they are only listening to the teacher 
talk; the only time the teacher is not using some materials is when 
she or he is talking directly to the students without any props or 
equipment to help convey her or his message. Materials include books, 
overhead projectors, records and record players, maps, blackboards, 
fancy programmed instruction kits, home-made (by the teacher) worksheets. 

Procedures . Procedures include any routine established by the teacher 
for the class to facilitate any activity that must be accomplished as part 
of the school day, but which is not directly related to performing academic 
tasks. A procedure is a set of guidelines about how to do something that 
has to be done on a regular basis . The most important characteristic of a 
procedure is that it has been planned and presented to the class as a way 
of doing something, and can supposedly be carried out without teacher 
direction on every occasion. The establisKment of - procedures by the teacher 
in order to get routine details taken care of is an important component of 
classroom organization. Therefore, one important objective of this study is 
to ^see what procedures . teachers do establish and what they have to do to get 
them working. , ' ' ^ 

^ Procedures may include collecting milk money, PTA money , getting pencils 
sharpened, paper and books distributed in the morning, moving the students 
through transitions , getting the students' work turned in, etc. 

Rules . Rules are similar to precedures in -that they are established 
agreements about behavior in the classroom, but they are more often definitions 
of what not to do,, and. they are not formed in response to necessary daily 
routines, as are procedures. Rules may include when talking is allowed, when . 



it . is appropriate and inappropriate to movd around the classroom, no fighting, 
no chewing gam, etc. ' 

Contacts . In this study contacts are defined as interactions between a 
teacher and an individual ' child . Three 'types of contacts, which can be described 

and discussed are: academic, behavioral, and procedural. 

; / * • 

Academic contacts are interactions' about some academic content. This 

' .' • • 

yfficiude^,guestion4 to the student and; information supplied by the teacher in 
."''^ . ■ ^ 

response. In orcjer for a contact to be considered academic, it must deal with 

the content specifically and not the student's work hatjits or behavior . 

Behavioral contacts occur when the teacher corrects some child for a 

misbehavior. / 

• ■ . / / 

Procedural contacts include all interactions about how to go about doing 
■ • / ~ 

something in the classroom which don't result from some student misbehavior. 

Transitions . Transitions are intervals of time between academic activities 

in which the primary activity /is moving from one thing to another. This may 

/ . ■ ■ 

include actual physical movement of students or it may be a matter of replacing 

some materials and getting out something else. Transitions can occur within 

■ ■"■ \ 

the room, or they may involve movement of students in and out of the room.. 

' Interruptions . In this study, interruptions are defined as events which 
- distract the teacher and /force ]hiis .or her attention (and often also the classes 
attention) on some unplanned event. This might include a messenger coming into 
the room from the office, an announcement over the loudspeaker , a fire drill, 
a fight in the room between two students, and a student getting sick and being 
taken to the office. / 

"Dead Time". This is an interval of time in which a student or students 

/ . . . 

apparently have nothing that they are supposed to be, doing. They are either 

/ ' ■ ■ .' • . 

between activities^4 or have been left temporarily by the teacher with no - 

/' 

provision made for their becoming involved in an activity. 
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CLASSROOM ORIENTATION 



Structural orientation 

What efforts have been made to organize the classroom 
environment? (10-15 minutes prior to the start of the 
first day) Bulletin boards used? How is furniture 
arranged? Map? General appearance of room? - 

0 

Personal orientation 

Introduction of students to a) teacher, and b) students. 

Procedural orientation 

(Consider, for example, procedurer elated to: a) classroom 
behavior, b) use of facilities, i '-and outside room, 
c) procurement, use and maintenance .. , materials, and d) general 
classroom activities,) 

What procedures and/or rules does the teacher outline for the 
students? 

How does the teacher orient the students toward the procedures 
or rules? . 

(For example: 

^ — How did the teacher introduce the procedure or rule 

to the students? Be specific in describing what she 
or he said. , ^ ' • , 

--Was the procedure or rule es.tablished in response to 
a problem that arose,' or was it done as a matter of 
course^ before any prob^lems occurred? 

— After introducing the^proceddre 6r;,.rule by explaining 
it, what did the teacher do? Was 'tliere "involvement 
of the students then in discussion o:^ it or practice of i 

— How well was; the procedure or rule followed the first 
* time? What did the. teacher do after its use for the 

first time? 

How does the teacher orient the students to the daily schedule? , 
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INSTRUCTIONAL EPISODES 



Materials 

Description _ 
What' types of materials are used in this episode? 

(i.e., multisensory, homemade, commercial, media format used, etc.) 

•Cp what extent are the materials identical for each student? 
(i.e., do all students have the same materials, are they all 
equally familiar with the materials, etc.) 

Presentation/Use 

What instructions are given (and who gives them) for use of the 
materials? 

Did there seem to be any problems associated with using these 
particular materials?. 



Instruction 

Beginning 

How are the students organized for instruction? 

(Whole Class — All students are attending to the same 
stimulus at the same time at the same rate. 
Group — Students are divided into separate groups with 
the teacher moving between groups. 

Individual — Students involved in independent work with 
^teacher generally available for help. 
Other — Combinations or unique situations.) 

How does teacher indicate cstart of activity and move students 
into position?. 

r — Systematic? 
— Students used? 
— Time use'd? (delays?)] 

What is the general subject for the lesson? What is the specific- ' 
content? (i.e., math — long division) 
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Mlddle 

What method is chosen for imparting the information? 

(discussion, lecture, group presentation, other?) 

What interaction patterns occur within the lesson? 

(Are -Students selected randomly, in any specific 
order, or are volunteers chosen? Are any sections 
of the class ignored?) 

Is the teacher sensitive to student attention span and comprehension 
of subject matter? 

Whaf'is the effect of the above, on the lesson? 



End 

What is the method for concluding the lesson? 

(test, summary, disorganized breakup, etc, ) 
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BEHAVIORAL CONTACTS 

Isolated or Specific Behaviors 

What behaviors does teacher define as on- or off-task? 
Teacher response to these behaviors. 
Consistency of teacher behaviors. 
Student reaction to teacher behaviors. 

Interruptions 

Classroom-based (behavior problems, sickness, etc.) 

( — Cause 

— Teacher reaction 

— Student reaction, etc.) 

Outside of Class (bell, fire drill, person entering, etc.) 
( — Cause 

— Teacher reaction 

— Student reaction,' etc.), 



23 

11-15 



Lainbert 

August 23, 1977 

Notes on I^it±[ig Clear ^ Concise, and, Specific Observational Narratives 

1. Select a vnriting and editing approach which is most comfortable and 

natural for you before, you go into the classroom. If necessary, exper- 
iment with approaches deliberately during the first few days, but again, 
decide before you begin actual observation. You will have enough 
to do in writing the narrative, keeping the tijie log, tallying engage- 
nvent ratings, and conpleting the Teacher Checklist, 



2. Use materials like the Narrative Guide as a focuser so that you attend 
to and write about only those classroom events which have to do with 
classrocm organization, ' Study and re-study the rnaterials which 
tell you the purpose and focus of the research project — especially 
the Narrative Guide and the Questions to Consider after VIriting the 
Narrative — until this framework is a mental reflex. The narratives 
must be focused and specific in order to be useful; they should not 
resemble creative writing or stream-of -consciousness . 



EKLC 



Be as concise and direct as possible without sacrificing conpleteness , 
Cut out "deadwood" and repetition — remember that you have both a 
deadline and a space requirement, so 'that each classroom anecdote should 
be a short story, not a novel. Keep /to the point with each anecdote: 
clear writing that explains and describes an event so that it can be 
used as data for further analysis should be both unified and coherent . 

Each aneodote is concise and direct 

but , also has unity (everything you write about it 

ties together thematically and 
logically) 

and coherence (everything you write about in ties- 

together sequentially or in some order) 



Be as specific and concrete as possible. Think of yourself as an inves- 
tigative reporter, so that each classroom anecdote, in order to be use- 
ful as data, needs supporting evidence . Supporting evidence is not 
lengthy overwriting, but specific detail. 

Ex. The teacher next explained housekeeping routines in a rather 
disorganized manner. 

What — exactly did the teacher say? 

How — did the teacher say it? (Paralanguage is as inportant as language,) 
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Notes on ^-ft ritljiq Clear, Concise, and Specific Observritional Narratives 
page 2 

Why— was it disorganized? (what were the children doing? were there 
interruptions?) 



5. Be_as careful as possible to distinguish between your objec tive reporting 
and your subjective in terpreting . The invisible or unobtrusive observer 
has Its counterpart in inpartial or clearly objective observing. 
By Its very nature writing a narrative is more subjective—i.e. , qualita- 
tive or evaluative—than caitpleting more objective or quantitative check- 
lists and other forms of coding. But you can separate clearly judgmental 
statements fron the rest of your narrative through bracketing. You can 
also gi^ard against purely subjective comments by supporting them with 
specific and concrete details. 

I ' . ^■ 

In thinking about objectivity versus subjectivity, reitember the great 
usefulness of the narrative to this study. Not .tiuch is known about the 
first weeks of school when teachers transform classrooms with twenty-five 
unorganized children into places Wiere efficient teaching and learning 
regularly take place., The guided narrative can capture an enomous 
amount of information, data vdiich might be lost ty a nore limited- and 
rigid coding sheet. Since this study will use both narrative (qualitative) 
and quantitative measures, observers hopefully will be able to capture 
the whole classroan—both the yoke and the white. 

Filially, here is one of the most sensible and accurate descriptions 
which contrast a more quantitative and a mof e qualitative method for data ' 
collection. Cairpbell (1974) describes the potential power of ireasures 
like observational narratives: 

For quantitative read also scientific; scientistic, and 
.naturewissenschaftlich. For qualitative read also human- 
istic, humanitistic, geis teswissenschaf tlich , experiential, 
phenonenological, clinical, case study, field work, partici- 
pant observations, process evaluation, and ooimon sense 
knowing. 
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- - WORD LIST 

The following short list of words is mea.it merely to assist the 
coder in writing classroom descriptions and narratives. The list is 
meant to encourage word choice and thereby to sharpen both observation 
and writing skills.' The list is by no means exhaustive and is, in 
fact, intended less as a reference than as a heuristic tool. 



assign 


dis ci^jline 


was ted 


lecture 


disorganized 


ill—prepared 


introduce 


lost 


3 prepared 


dpmons trat p 


chAOQ 

i A CL KJ ^ 


structured 


ilTust'Tat'e 


hn^ t"i 1p 


ToospIv— s tmcturpd 

w ^ y w Lm ^ Im X. V4 


organize 


w ^ ^ ^ V ^ 


planned 


order 


polite 


p Ian 


threaten 


cooDprAtive 


fidget 


repeat 


helpful 


chatter 


accept 


fair 


giccle 


rej ect 


warm 


loud 


diirect 


nice 


rowdy 


divert 


tolerant 


1 misbehavior 


control 


bored 


restless 


correct 


sleepv 


misbehave 


disruDt 

w ^ U fc^ w 


easer 


procedure 


compliment 


clever^ ^ 


rule 


praise 


anxious 


suggest 


reward 


attentive 


oral 


wander 


rpsDonslve 

^ w w & A w V ^ 


talkative 


whisnpr 

W A A w w ^ ^ 


X. J. L> J. W CL wim 




laugh 


humorous 


unhappy 


cri tici7p 

^ ^ M w v,^ 


creative 


noisy 


embarrass 


gentle 


friendly 


ridicule 


vacue 


games 


discourage 


house-lceeping 


conscientious 


ehcourage 


seat work 


messy 


T*on T"*{ man/I 
Ju C ^ i. XUidiiU 


cunxus eu 




exhort 


clear 


interesting 


cotnmand 


style 


strict 


promise 


accomplish 


brusque, brusk 


exclude 


effective 


group 


ignore 


ineffective 


circulate 


point 


effort 


consult 


discuss 


outburst 
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NOTING TIME INTERVALS ON THE NARRATIVE RECORD 



Since time is an important focus of this study, one very important task of 
the observer is to note the length of transitions and dead time when they occur 
in the classroom* 

On the narrative record form, the two inches of space to the left of the 
numbered lines are to be used to note times as described below. ' 

Noting beginning and ending times on pages . On the first line of the first 
page, where labelled "Start," the observer should note the time when beginning 
the narrative. At the beginning 6f each new page, the time should also be 
noted. At the end of the last page, where labelled "Stop," the observer should 
note the time when the narrative was completed. (This should go on the last 
line even if the narrative doesn't fill up all of the lines on the page,) 

Noting beginnings and ends of transitions and dead time . Whenever the 
observer is aware of a transition or dead time in the class, he or she should 
note it under the column labelled "Beg" as the time when the interval started, 
and a T for transition or a ^ for dead time. (For example, T-9:27,) When 
either is .completed, the observer should note the end time in the same way 
under "End," The line on which either time is nqted should correspond to the 
narrative record where the transition or dead time is being described. 

Obviously, there is much observer judgment involved in determining when 
to start and stop. timing. Discussions of each use of time are presented below, 
but tfie observer should remember that there will be unclear situations. When 
these occur, note and describe in the narrative any ^circumstances which made 
it difficult to define the beginning and end times accurately, or which made 
it difficult to arbitrarily define an interval as a transition or dead time. 
Whenever in doubt, time events as you think is appropriate, by noting as many 
times as might appropriately designate the beginnings and 'end, and discuss it 



with someone back in R&D. You will eventually want to turn in a set of matching 
beginnings and ends for a certain number of transition and dead times, but if 
you cannot mak^ that decision on the spot, then record as much information as 
is necessary to.discuss J.t and decide later. 
■ Transitions are intervals of time between academic activities in which 
the primary activity is moving from one uhing to another. This may 
include actual physical movement of students or it may be a matter of 
replacing some materials and getting out something else. Transitions can occur 
within the room, or they may involve movrraent of students in and out of the room. 
Some examples of transitions are: the seudents arriving in the morning and 
getting "settled in"; the students lining up to leave the room, either as a 
whole class or as a group (the informati vx abovat the part of the class involved 
would be entered in the narrative as "whole-^class , " middle math group, etc.); 
the students moving between activities vrithin the room; the students putting 
away some materials and/or getting out new njaterials, whether moving or staying 
at their seats. The beginning point of the transition would be the signal 
that is given to the students to begin. Usually this is clear and comes from 
the teacher, although school bell.^- li^y also be used. Remember to describe the 
activities which occur during the transition . The end of the transition is 
not as easy to determine. When it terminates in the teacher beginning a new 
lesson or activity which was the objective of the transition, then this shoula 
be considered as the end. When this, does not occur, the observer > should use 
as the end point that time when the teacher would define the activities as 
starting. When some students have gone through a transition and performed the 
appropriate motions (such as moving from one thing to another or getting out 
materials), but do not begin the appropriate activity after doing so, the 
observer should still consider the transition as complete but might note that 
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a certain nun>ber of .Individuals had "gone through -the motions" but were not 
actually beginning the new task. 

■ Deal^ is an interval of ti.e in which a student or students apparently ' 
have nothing that they are supposed to be doing. They .re either between 
activities, or have been left temporarily by the teacher with no provision 
n^ade for their beconang involved in an activity. To be considered dead time 
and noted as such, a significant part of the class (5 or .ore students) sho_uld 
be involved. (Il^erefore. a single student who is at loose ends would not be 
timed, although the observer might wish to note this in the narrative if it seems 
important in terms of the teacher's organization.) If dead time has an obvious ' 
beginning, such as an interruption which distracted the teacher, then this should be 
noted as the beginning time. The ending time would be the teacher's return to direc- 
tion of the students. If the beginnings of dead time is not. obvious, then the 
observer should note the time at which it became obvious that at least five 
members of the class had nothing definite expected of them. Again, the end ' 
time would be some definite direction from the teacher. Remember that the 
critical aspect of dead time is that the students have not been given anything 
to do. Therefore, students who are working on assignments and whose teacher 
is called away are not left in dead time. Likewise, if thi teacher organizes 
something on the. spur of the moment, even a game, then they are not in dead 
time. Needless to say. the teacher's action in each case should be described. 

OBVIOUS REMINDER: If you rewrite your narrative on return to, R&D. please 

make sure that the times get transferred according to. 
• the activities being described in the narrative, and not 
according to the lines used on the rough version. 
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Date 



School II 
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Observer // 
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STOP! 
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FOLLOW-UP' QUESTIONS FOR Ni\RRATlVE RECORD 

A. QUESTIONS TO ANSWER FOR OBSERVATIONS CONDUCTED ON DAYS 1-5 (FIRST WEEK) 
!• \>rtiat was the room arrangement? 

2. \^mat materials/assignments were ready for use in academic activities? 

3. How did the teacher introduce himself or herself to the class for 
the first time? 

4. How were students introduced to one another for the first time? 

5. How were students introduced to classroon rules and procedures for 
movement or noise, use of materials, following schedules, 
behavior, etc, ? 

6. Did the students participate in making the rules for the class? 
7* Were the consequences of misbehavior discussed? 

8. I^at beginning-of-the-year paperwork was handled in class? 

I. BEGINNING OF ACTIVITY/TRANSITION INTO ACTIVITY OR SET OF ACTIVITIES 

9* How did the teacher indicate the start of the activity or activities 
and x^7hat did he or she do to get everyone moved into, position? 

10. How did the students react to this? 

11. What did the teacher do during the transition? 

12. If there were delays, what caused them? 

13. \nien the tfeacher had a group together (either large or small) and 
began a group activity, how was it introduced?' Was there an 
overview or advance organizer? 



14. 



WHEN THERE WAS INDIVIDUAL WORK: 

Did the ^teacher decide what the students would do for individual 
work, or was there student choice of assignment? 



15. If student choice, describe what happened. 



16- 



17. 



If the- teacher had decidei, how did the students know what they . 
were supposed to do? * 

Was there evidence of posting or assignments, posting of who went 
where, etc.? 
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Follow-up Questions for Narrative Record-Continued 



STUDENT ACTIVITY OR ACTIVITIES 

18. What were desired* student activities? (To be answered on chart at 
beginning of question packet. Elaborate in space provided if 
necessary for complete understanding.) 

19. Were all of the students doing the same' activity most of the time, 
or were there different activities going on in the room at the 
same time? 

20. What were the major problems encountered' by thc^ teacher in terms 
of not getting the students to do what was desired? 

21. ^fliatwere the teacher's responses to unsanctioned behavior? Did any 
of the following occur enough times that it stands out in your 

.memory as typical of that teacher's style in responding to 
misbehavior? 

a. ignoring 

b. citing rule 

c. ordering student to' stop the misbehavior 

d. signaling appropriate behavior 

e. expressing feelings about misbehavior 

f. questioning student to gain information about misbehavior 

g. isolating, separating 

h. punishing in some way . 

22. What were the typical results of the teacher^ s reponse to misbehavior 

23. Was tlie teacher consistent in his or her response to misbehavior? 
CONTACTS WITH THE TEACHER/ FEEDBACK ON WORK 

* '^'^ 

24. Did the teacher seem to have a system for contacting students? If 
there was no^ apparent system, how would you describe his/her style 
in selecting students for ^interactions ? 

25. For each activity engaged in by students' was there a product or 
assignment which reflected what the student had done during the time? 
(To be answered on chart at beginning of question packet. Elaborate 
in space provided if necessary for complete understanding.) 
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Follow-up Questions for Narrative Record-Continued 



26. Was there a set ' procedure for turning in work after it was done by 
individuals? If so, describe its use. If not, what did the 
students do with their work? 

27. If there was not an assignment turned in, how did the teacher find 
out what the students had done during ithat period of. time? 

28. How did the students indicate that they 'needed help? What did the 
* teacher typically do in response to their signals? 

29. Were provisions made for what to do when assigned work was 
completed? Did any problems arise because of students completing 
work and then having nothing to do? 

WHEN THERE WERE SMALL GROUPS AND INDIVIDUAL WORK 
GOING ON SIMULTANEOUSLY:- 

I^at did the teacher do when students approached him or her while 
teaching the small group? Was there a consistent response? 



30. 



31. Did the teacher seem to be monitoring the rest of the class when 
working with the small group? How? 

32. When the t&acher interrupted himself or herself to deal with 
something out-of-group, what were the reasons? 

33. Did the teacher leave the small group to deal with something in 
the rest of the room? What did the students in the group do 
when this occurred? 

34. Describe the seating of'^the students in the small group, the ' 
teacher, and the out-of-group students with respect to each other. 
Say which way everyone was facing. 

35. What happened if out-of-group students needed help while the 
teacher was with the small group? If they were delayed, how long 
was. It before they got help? 

' ' ' >^ . . 

PROCEDURES' AND RULES' IN EFFECT FOR THE ACTIVITY OR ACTIVITIES 

36. What procedures were in- effect during the activities? Include 
monitors, reward system and/or behavior mod system, planned 
policies for teacher contacts, etc. 

37. For each procedure, describe its functiolning: How well did it 
accomplish the purpose of getting routine . activities accomplished 
efficiently? 
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Follow-up Quoationa for Narrative Uacorcl-Contlnuotl 



o 

38. \^iat procedurea were eatabUahecl for the flrat time? For each, 
answer the following questiona. 

a. How did the toaclior introduce the procedure to the atudenta? 
Be specific in describing what he or she said, 

b. Was the procedure established in response to a problem that 
arose, or was it done as a matter of course before any 
problems occurred? 

c. After introducing the procedure By explaining" it, what did 
the teacher do? Was there involvement of the students then 
in discussion of it or practice of it? 

d. How well Was the procedure followed the first time? What did 
the teacher do after its uses for the first time? 

39. \Jhat rules were established for the first time? For each, 
answer the following questions. 

a. How did the teacher . introduce the rule to the students? 
Be specific in describing what he or she said, 

b. Was the i^ule established as the result of a problem or was 

it done as a matter of course before any problem arose? ^■ 

c. After introducing the rule, what did the teacher do? Was 
there a discussion with the students or any other kind of 
involvement? 

d. How well was the rule followed that day? When someone did 
or did not follow the rule, what did the teacher do? 

40. Did the teacher remind the students about any rules or procedures, 
and/or reexplain any? What happened before this reminder or 
reexplanation? Describe the teacher's reaction. 

'f _ / , ■ - - 

41. What violations of already established rules and/or procedures 
occurred that were not responded to by the teacher? 

42. \^at other behaviors occurred which were not responded to by the 
teacher, but which struck you as inappropriate for the classroom? 

F. MATERIALS 

For each of the materials used during the activity or activities, answer the- 
following questions*: 

43. Was material set up and ready to use at the beginning of the 
activity? If so, how had this been accomplished? If not, what was 
done to get the materials distributed or set up? 
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Follow-up Questions for Narrative Record-Continued 

A4. How wa3 the material supposed to be used or what was its intended 

effect? (Answer in terms of student activity.) (To be answered 

on chart at beginning of question packet. Elaborate in space 
provided if necessary for complete understanding.) 

A5. What was the actual use by and/or effect on the students? 

46. When the use and/or effect fell short of the ideal, what factor(s) 
contributed to the problem? Please be very specific, and indicate 
what factors mentioned were beyond the teacher's control. 

47. What ins tructiOBS were given to the students at this time about 
the use of the material? 

48. What instructions had apparently been given to the students about 
the use of the materials before this time? 

49. I^at procedures were in effec^, regarding the use of the material? 
Include anything about getting it out, using it in an activity, 
and putting it up. 

50. Give any anecdotes that indicate how the use of any materials might 
have affected the teacher's frequency of contact with a student. 

51. Give any anecdotes that indicate how the use of any materials might 
have affected the time and attention a student spent on task. 

G. INTERRUPTIONS MD DEAD TIME 

What interrupted the flow of activity and/or required the teacher's 
attention unexpectedly? Be very specific and describe any factors 
which were outside the teacher's control. Specify source as 
internal or external to classroom. 

For each of the interruptions, describe the teacher's response. 

What was the result of the teacher's response for the majority of 
the class? That is, what did they do while the teacher was dealing 
with the interruptions? 

How much time and how many instances would you classify as "dead*' 
time that have not been described above for any part of or for all 
of the class? 

When there was dead time, what preceded it? In your opinion, what 
caused it and what could have prevented it? 

What did the teacher do during the "dead" time? 

How did the teacher indicate the end of the activity and the need 
for a transition? 



52. 

53. 
54. 

55. 

56. 

57. 
58. 
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Follow-up Questions for Narrative Record-Continued 



59. Was there a summarizing statement at the end o- the whole-class 
activity and/or any reference to future activity which would 
follow? (Include assignments,) 

60. How much advance notice was given to the students so that they 
could start to finish up their work and put away materials? 

61. How did the students respond to the teacher's signals as far as 
finishing up/cleaning up before moving on to the next activity? 
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SAMPLE \FORM: FOIXOW-UP QUESTIONS FOR NARRATIVE RECORD 
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Procedures for Measuring Student Engagement 

Since one of the criteria of a "smoothly-running" classroom is that 
the students are engaged in academic, on-task activities as much as possible, 
one of the measures taken in this study is the degree of student engagement 
at several times through the day. 

To do this, you will look up from your note-taking eyery 15 minutes, 
scan the classroom, and count the number of students who can be classified 
in each of the five categories described below: 

!• Academic-on-task ^ Students counted here are those who are engaged 
in appropriate activity which is directly related. to learning the academic 
curriculum* Thi? includes listening to the teachers, answering questions, 
doing written assignments, participating in activities at learning centers. 
The definition of appropriate activity is what the teacher would expect 
of that student at that time and would indicate approval uf if asked . 
(However, you are not judging whether or not the student is answering correctly 
b.ut whether he or she is directing attention toward the appropriate stimulus, 
therefore, answering a teacher question incorrectly is still appropriate 
behavior, because the student was listening and responding when expected). 

2. Procedural on-task . Students counted here are those engaged in 
.activities whiqh are indirectly related to learning of the academic content, 
and which have been deliberately set in action by the teacher for the purpose 
of getting routine details of the day accomplished. These are the details 
which must be attended to in order to move on to academic activities, and 
which occur with deliberate teacher sanction. Examples are : participating in 
roll call, PTA money collection, other money collection, lining up for 
transition out of room, moving about the room for transition between activities, 
performing a monitor job for the teacher.. Students counted in this category 
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would be considered by the teac.hL to be doln^ . xactlv what .h.y are suonos.d 
to be doing at that time, an d thlr ta sk Is- a procedural one, not an acad^^f. 



one , 



3. Off-task, sanctioned. Students counted here are not clearly engaged 
In either of the on-task categories, but are doing something which Is still 
appropriate .and accepted by the teacher. The activity, however, has not been 
Initiated by the teacher, and many Include the student attending to personal 
needs by Interrupting some on-task activity In an appropriate way. Examples 
are, waiting at the desk to see the- teacher, sharpening pencils during the 
seatwork period, going to the restroom during a lesson (not as part of an 
organized group), talking quietly after finishing an assignment when the 
teacher allows this. The teacher w ould not define "off-task. sanr^^nn.H" 
activities as bei ng "..xartly what the student is supposed to be doln^ a^ 
that tlae" bu t the teacher would not consider the behavior totally 
inappropriate either . 

^- Off-task, unsanctioned. Students classified in this category are 
clearly not engaged in an on-task activity, but are instead doing something 
that the teacher would not approve of. Such activities may not lead to 
teacher correction , but It Is clear that if the teacher was asked about the 
student's activity, he or she would sav t hat it was inappropriate and not 
desired. Examples include: Inattention during a lesson or while doing 
assignments, being distracted while doing assignments, talking or moving 
about when not allowed, failing to participate in a transition when this is 
expected, breaking any classroom rules which are obvious to the observer. 
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5/"' Can*t tell . When the observer cannot make a clear judgment as to 
which of the above categories is appropriate for a student, fjien that student 
should be counted here. This category should not be used to avoid making 
decisions, however, since every attempt should be made to classify the 
students somewhere. How ever, when the observer feels that two or more 
categories seem equally appropriate because not enough information is 
available, then the student should be counted here. 

Summary 

1. Every 15 minutes, starting 15 minutes after arriving in the classroom 
and beginning observation, the observer should scan the room and count the 
number of students who can be classified in each of the 5 categories. 

2. The determining principle for classifying is what the teacher 
considers appropriate and is expecting of the student at that time. 
Using the form to indicate student engagement 

Attached is a form to use in the classroom while completing the student 
engagement counts. 

Each numbered column represents the end of a 15 minute interval and should 
be used to record the number of students in each category at that point in time. 

If you miss counting at the time you are supposed to and do not remember 
until you are more than 5 minutes to the next scheduled time, then note on 
the form that the count was missed, and wait until the scheduled time to do 
the next one, noting on the form in its appropriate place. 

Whenever you do the count, note on the nari;ative record form in the column 
directly to the left of the line numbers where you were in the narrative when 
you did the count. Do this by noting the number of the column on the student 
engagement form. (1 for first count, 2 for second, etc.) 
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Complete the information at the top of the column for each count as 
to the subject matter content, and activity which were occurring at that 
time. This Information should be retrievable from the narrative, and can 
be filed in on the student engagement form at a later time, if the observer 
knows that it is written down. 
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Teacher # 
Date 

Subject 
Content 
Activities 



School »^ 
AM PM 



:>Aru'Lii FORM: 
STUDENT ENGAGEMENf RATING 

Observer if 



* students present 

Page of 




ERJC '^ Ten 



Reliability of the Student Engagement Ratings 

The primary short term criterion used in the Classroom Organization 
Study to identify effective teachers is the degree to which their 
students engaged in on-task behavior. Accordingly, observers made 
estimates of the frequency of students who were on-task and off-task 
during their visits to the classroom. During each observation, ' at 15 
minute intervals, observers noted the number of children who were 
on-task or off-task. A check pn the reliability of the on-task/of f-task 
variable was made by examining the scores for each teacher during 
approximately 12 hours of observation the second and third weeks of 

. o 

observation. Frequencies of students rated as off-task each time the 
assessment was made ranged from a low of approximately one pupil rated 
as off-task on the averafee to as high as ten pupils rated as off-task, 
the median number of children rated as off-task each time was 
approximately ofour. 

An estimate of reliability was derived by computing the off-task 
behavior scores based upon odd and even numbered observations and then 
correlating the two. The Spearman-Brown formula was used to estimate 
the reliability of the total set of observations combined. This 
procedure resulted in a reliability of .89 for the of f-task/on-task 
variable. 
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GUIDELINES FOR USING THE COMPONENT RATINGS 
Designing Les^^ons to Achieve Particular- Purposes 

a. Describes objectives clearly— Has the teacher indicated what the 

students are to learn during the lesson? Look for indications 
of this in materials given to the students, written on the board 
or overhead projector, or listen for it when the teacher Is intro 
duclng the lesson. It should be clear what the students are 
expected to know or tc be able to do as a result of participation 
in the lessons. 

b. Activities are available for different children — Not all children 

are expected to perform the same task. In other words, there are 
differentiated assignments or activities within the same lesson. 
Lessons a re built on the curriculum and instruction from preceding 
activitie s in the unit / The relationship between the lesson and . 
previous lessons in the same subject area should be apparent and 
can be inferred from comments by the teacher or questions from 
the students. In order to give a high rating .in this category, 
the observer should have direct evidence that the teacher has 
incorporated prior material into the lesson. The point of this 
category is to differentiate teachers who treat each topic as a 
new, separate topic, versus tho^e who develop integrated series of 

V 

activities and lessons with long range goals. 
^- The attention span s of the children are considered in the design of 
the lessons. Evidence that attention spans have been taken into 
account include provision for varied activities, things to do if 
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children complete their work early, rest breaks, lessons that are 
not excessively long, and variations in terU^f.pg style to arouse 
interest /attention. 

Locating, Constructing, and Using Materials • 

^- Uses a variety of materials. During a lesson or activity a teacher 

may use numerous media and' materials, or may restrict the activitie 
to a single set of materials. Generallv, the minimum set of 
materials that will be used will be a workbook, textbook, or 
ditto handout accompanied by verbal teacher explanation and the 
blackboard or overhead proi ector -presentation. Other materials 
or media Include movie pt. , tape recorders, audio cassettes, 
manipulative materials, games, and supplementary reading materials, 
as well as teacher-made or pupil-made materials.' Rate a 1, if the 
minimum set of materials is characteristic of most lessons. 
Rate a 2 if the teacher incorporates a variety of materials 
regularly, and rate a midpoint if some variety is evident but 
only in some lessons. 
^' Materials are ready a nd available in sufficient quantity— Rate a 7 
if all materials are ready on all occasions.' Rate a 1 if the 
teacher continuously runs but of materials or spends a lot of 
time hunting them up and getting them into pupil hands. 
The materials support instruction . The intent of this category is 
to differentiatfe'the use of materials simply as time filler or to 
get through the period versus their integrated use with the 
Instructional objectives of the unit or lesson. It should be 
obvious to the observer how and whether the materials support the 



' • . ' , Page 3 

objectives; if not, give a low rating; 
d. The use of materials as accompanied by clear directions . Look for 
instructions given verbally by the teacher and repeated by the 
students, and written instructions either on the blackboard, 
overhead projector, or in handout form. Also, an indication of 
clear directions can be obtained by the ease with which students 
begin their use of the materials, and the frequency of questionss 
and repeated directions issued by the teacher. 

Presenting Information 

a. The teacher has distracting; mannerisms or characteristics . • A distract- 

ing mannerism is some gesture, vocal quality, or behavior that 
causes the observer to be distracted from some aspects of the 
lesson. . The observer will have to judge whether the behavior is 
distracting to the children, and whether it~continues to be dis- ^ 
tracting after a period of time. 

b. Eye contact . Does the teacher keep her eyes on the class and does 

the teacher make the rounds of the classroom with her eyes? She 
'jhould, avoid watching just a few students and inistead maintain 
visual contact throughout the lesson. 

c. The teacher's presentation was clear . Questions and instructions 

were presented in a coherent sequence, adequate examples were 
provided, and skills, when taught, were appropriately demonstrated. 

d. Presentation was adapted to different levels . The teacher related 

information to different ability levels, used a variety of 
approaches if the content was not initially comprehended, used 

r O 
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appropriate vocabulary, and paced the lesson sufficiently to 

allow for a thorough coverage. 
G^ves or seeks ratlon.lp fn. ^se of principles or rulp. . The 

teacher Is careful to explain reasons why certain procedures, 

rules, formulas, etc., are used, rather than simply presenting 
them to the children as the way to do it. The teacher's questions 
encourage analysis and reflection by the students. The teacher 
asks students to explain or justify their conclusions, or to 
give reasons or background information. The teacher explains 
why she engages in certain activities, seeks adequate information 
before forming conclusions. 

Developing Positive Arrirnde s, Including Attitudes Toward Self, .nhj ^.. . 
Matter, and Others 

^' Ihe_teacher states desired The teacher who receives high 

ratings on this characteristic will be one who frequently identi- 
fies appropriate behavior or attitudes, such as respecting other 
students, and'valuing their contributions and opinions. Such ' 
things may be observed by lists of rules, displayed in the class- 
room as well as through verbal instructions given by the teacher. 
Lower ratings would be indicated by the absence of these charac- 
teristics as wel]_as^ by rules that were framed strictly in " 
negative terms. 

b. .Deg ree of pup il sucress - Students appear to be able to complete 
successfully the assignments and tasks the teacher gives. 
Discussions are participated in by more than a few pupils, and 
questions which the teacher asks are either answered correctly 
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or the teacher stays with the pupil ' 
Lessons are related to pupil interest o 
this characteristic can be displayed 
teacher when she makes reference to 
being studied and aspects of the chi: 
Other relevant information may be ob 
materials used by the teacher, or lei 
are clearly taken into account, such 
describe parents' occupations, trips 
instance of this type of behavior is 
contributions of different groups of 
those groups are present in the ciasj 
Pupils are given clear expectations witl 
standards , Evidence of this characti 
verbal statements on the teacher's pi 
when children are expected to turn ii 
complete form. A reasonable amount c 
ness is emphasized -in the finished wc 
observer can note whether students ai 
materials sorted carefully and in ret 
teacher emphasizes respect for propel 
and whether the rules that are postec 
to the children. 
The teacher's behavior shows positive re 

c 

children . This characteristic can be 
behav^iors on the teacher's part, inc] 
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children in a friendly manner before school begins, vhether she 
takes the time to talk to them in a friendly and interested manner 
about their concerns and interests. Note also the degree to which 
. the teacher touches and gets physically close to the children. 
These behaviors do not need to be in response -to any specific 
activity or behavior of the children, but rather should reflect 
the degree to which such regard appears to be spontaneous and 
natural on the teacher's part. 

Managing PuT>il Behavior 

A. variety of rewards are available for appropriate pupiJ. behavior . 
Rewards that may occur in a classroom may include social reinfor- 
cers such as' teacher praise, displays of good work, peer approval, 
stars, and other recognition for appropriate performance. In 
addition, concrete reinforcers such as gifts or tokeiLs may be 
employed. Finally, activity and privilege reinforcers. such as 
earning points or checkmarks to gain a privilege, being allowed 
• to go to the library, or leading a group to lunch, running an 
errand for the teacher, going early to some activity, being given 
free time, using a game or activity area, all constitute potential 
reinforcers in the classroom, 
b. Gives signals for appropriate behavior. This class of behaviors 
refers to any activities, both verbal and otherwise, which the 
■ teacher uses to let students know that they should begin behaving 
in a particular manner. Some typical signals include using a 
bell to signal time to begin an activity, lights on or lights 
off , a sign with STOP and GO to control movement or noise level. 
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Teachers also will move around the room and use their presence as 
a signal for attending or other appropriate behavior. Verbal 
statements which orient the students toward behaving in a partic- 
ular mode are also signals. Examples of these include such 
phrases as: "Does everyone have their thinking caps on?" or \ 
"Let's have all eyes up front," or "I'm going to call on someone 
In one minute to answer this question, so everyone listen care- 
fully," or "Let's hear what Sally has to say now." However, 
orders or commands to pay attention directed at inattentive 
students will not be considered signals or cues for appropriate 
behavior. In other words, if the teacher seems to be trying to 
get an inattentive student to pay , attention by directly operating 
on that student, then such behavior would not be regarded as a - 
cue or signal but rather as reinforcing the inattentive behavior 
(the next rated category). The present category is reserved for 
those instances of teacher behavior, which are designed to elicit 
orienting responses from the students without singling out an 
individual in any obvious manner. 
Teacher reinforces inattentive behavior . Rate a- 7 if. there is a high 
amount of attending to inattentive behavior and this seems charac- 
teristic of the teacher. Rate a 1_ if the teacher does not reinforce 
inattentive behavior. Occasional reinforcement of inattentive 
behavior would receive a moderate rating pn the scale. By rein- 
forcing inattentive behavior is meant any teacher action which 
calls attention to individual 'students' .inattentiveness. We will 
consider inattentiveness to mean visual wandering, daydreaming, 
covertly avoiding engagement in a lesson, or any other activity 
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that represents pupil disengagement but is not overtly disruptive 
or interfering with other students. 
Disruptive Pupil Behavior 

On this scale you are to estimate the amount of disruptive behavior that 

occurs in the classroom. Disruptive behavior refers to any activity 
on the part of. one or more students that .prevents the teacher from 
continuing the lesson, or that elicits the attention of at least 
several other students. Excluded* from this definition are inatten^ 
tive behaviors and behavior that involves only one or two other 
students, such as one student whispering, writing notes, or 
goofing off. However, if the behavior elicits the attention, 
although not necessarily the involvement of niimerous other students, 
then it would be classified as disruptive behavior. A 2, ^^^ing 
would be obtained if such behavior occurs with a high degree of 
frequency. Use a 7_ to note a situation which is habitual and is 
a constant problem for the teacher and other students. This 
situation would be indicated by such behavior once everfy five 
minutes. A midrange rating would be obtair'^ed if such behaviors 
occur with moderate frequency such as several on the average per 
hour. A rating of 3^ would be indicated by the complete absence 
of any such incidents. Note that by disruptive behavior the 
behavior does not have to be as extreme as a knife fight. Rather 
it means any behavior that distracts or interferes with a signif- 
icant number (more than two) of students attending to their work 
or. the lesson. 

a. The disruption is stopped quickly . Whatever action the teacher takes 
causes the behavior to terminate-jwj^thout involving other students 
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or without serious interruption to the activities in the lesson. 
There is an absence of ripple effect and things return to normal 
rapidly. A _7 indicates that disruptive behavior was stopped 
quickly and a 1_ indicates that the behaviors tended to persist or 
escalate and that the teacher was often unsuccessful in terminating 
the behavior. The middle of the scale rating indicates that some 
behavior or some incidents were terminated whereas otheVs were 
not or that generally the teacher is only able to terminate them 
after some struggle. 
Rules are given . The high rating would be obtained by a teacher who 
generally deals with disruptiveness by citing classroom rules that 
prohibit the behavior or that identify appropriate alternative 
behaviors. If the teacher does not use this mode, then rate a 1, 
and if this procedure is used on some occasions, give a middle of 
the scale rating. 

Time out, isolation, or separation is used to terminate the behavior . 
Rate the degree to which this type of solution is practiced by 
the teacher. 

Teacher criticizes and/or justifies authority . This type of behavior 
is one in which the teacher attempts to terminate the behavior by 
criticizing the student for what he did. Criticism may be personal 
or may simply focus on the behavior. It involves more than simply 
citing a rule for appropriate or inappropriate behavior, however, 
in that the teacher is directly criticizing the student for his 
. actions. The intent of this type of behavior is usually to put 
the student on the defensive. Also, the teacher may justify her 
authority in this situation by expecting, by stating that the 

11-44 



Page 10 

student should do it because he Is being told to do it. Rate a 7 
If a high degree or amount of this behavior is present. Rate a 1 
if the teacher never uses this approach ro disruptive behavior. 

e. Conference . The teacher n..eets privately with the student or students. 

This may occur in the hall or in the classroom. Note whether the 
teacher's discussion is intende^ to be public or private since 
6nly private meetings will be considered conferences. A 7 indicates 
frequent use of ' this procedure _and a 1 indicates no use. 

f. Punishes. The teacher punishes pupil behavior in order to terminate 

it. Punishment may involve giving demerits, taking away privileges, 
or invoking some negative consequence such as requiring some ta::k 
to be carried out, or depriving the student of a desired activity 
such as recess. It may also involve requiring the student to 
stay after school. 

g. Problem solving . The teacher may deal with disruptive behavior by 

discussing it with a student or students in order to arrive at 
some solution that may require the student to change his or her 
behavior. The important thing to note here is that. ' tbere is some , 
genuine attempt to actually discuss the pt-obiem as opposed to 
simply giving the. student a single alternative. When problem 
solving is used, the teacher or the students may suggest a 
solution. Usually there should be some ladicaf.ion that the 
solution is acceptable and will be tried.. The teacher may also 
discuss the problem with the total class and ask for solutions 
from them even though not all men^bers of the class ate involved 
in the disruption, 
h. Ignores. The teacher makes no attempt to terminate the behavior. 
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Interacting Effectively V 

a. Listening skills . These teacher behaviors refer to statements which 
the teacher makes that allow a child to continue talking about 
some problem or feeling. By using such statements the teacher 
indicates an acceptance of the child's feeling or at least ^..ovides 
the opportunity to express it. Examples of such statements on the 
ttacher's part include, "Would you like to talk some more about 
it?" or "I*m not sure I understand; can you tell me more?" or 
"You seem upset (or hurt, or happy, or sad, or confused, or not 
sure, etc.)." The teacher may also demonstrate listening skills 
throuf>h nonvefbal behavior by hex posture and orientation toward 
a child who exhibits some feeling or problem, llie-TcS^Tth^ to 
look for fs whether the teacher's behavior enables the student 
to furth-r identify his feeling or to, explain it. Rate a 7_ if 
such behaviors saem characteristic of the teacher whenever feelings 
are expresp.ed by the students. Give a midpoint rating if such 
behaviors seem to be used occasionally by the teacher in response 
to such student expressions, and rate a 1 if the teacher generally 
does not respond to such statements' with appropriate listening* 
skills, or if the teacher attempts to terminate such behavior 
or. seeks to avoid it, 
b, Expr es se£;_f;-egUn^. In her interaction with the children, the 

teacher spates how she feels about certain activities or behaviors, 
•rhese expressions of feeling may be both positive or negative 

^ and may include such things as, "I am happy, sad, angry, annoyed, 
upset, disturbed, etc," Such expressions may occur during class 
discussions, behavior management situations, during procedural 
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activities, or when any aspect of student behavior or class 
activities is being discussed. It can be with individual 
students, or a group of students, or the whole class. The point 
here is to observe whether the teacher is overtly indicating 
to the students how she feels about various aspects of class- 

t 

room activity, such as their performance, their behavior, or 
their relationships and interaction with each other. It is 
important, however, that the teacher's expression of feelings 
not continually place the students in a defensive or vulnerable 
situation. A teacher who continually reacts to inappropriate 
pupil behavior by telling the students how angry , it makes her is 
not really expressing her feelings, but rather using them as a 
kind of battering ram to wear the students down. Appropriate use 
of this type of behavior is indicated when the teacher's expression 
of feelings appears to be a clear and direct attempt to inform the 
students about how she feels about some asp'ect of their behavior, 
work, or interation. A teacher who overdoes the "I am angry" 
routine is not using this skill effectively. Rate k 7_ when the 
teacher expresses feelings naturally and frequently, uses a mid- 
point for^^occasional expression of feeling, and a 1 for no expression 
of feeling or for inappropriate use of this behavior. 
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COMPONENT RATINGS 
Teacher 



No. Students Present 



Observer 
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// Date 



1. Lesson Design 

7654321 a. Describes objectives 

clearly 

7-6 5 4 32 1 b. Activities for 

different children 

7654321 c. Built on preceding 

C&I 

7654321 d. Attention spans 

considered 



7 6 5 4 3 2 1 



7 6 
7 6 



6 
6 
6 



7 
7 
7 
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5 
5 



4 3 2 



4 3 2 



1 
1 



2. Lo ca t ing ^ Con s t r u c t i n g , 
Using Materials 

a. Uses variety of 
materials 

b. Materials ready 

c. Materials effec- 
tively support 
instruction 



7 6 5 4 3 2 1 d. Clear diretti 



7 6. 5 4 3 2 



5 
5 
5 



7 6 5 



ions 

3. Presenting Information 

1 a. Distracting 
mannerisms 

1 b. Eye contact 

1 c. Presentation clear 

1 d. Adapted to different 
levels 

3 2 1 e. Provides /seeks 
rationale and 
analysis 



4 3 2 
4 3 2 
3 2 



6 5 4 3 2 1 



6 5 4 3 2 1 



6 5 4 3 2 1 



4. Developing Attitudes 

a. States desired 
attitudes 

b. High degree of 
pupil success 

c. Content related to 
pupil interest/ 
background 



7 6 5 4 3 2 1 d. Reasonable work 

standards 

7654321 e. Positive regard 



5. Managing Pupil 
Behavior 

5 4 3 2 1 a. Variety of rewards 

5 4 3 2 1 b. Signals appro- 
priate behavior 

7654321 c. Reinforces inatten- 
tive behavior 



7 6 

7 6- 



7 6 

7 6 

7 6 

7 6 



7 6 
7 6 



u J 



Disruptive Pupil 
Behavior 

5 4 3-2 1 a. Stops quickly 

5 4 3 2 1 b. Rules given 

5 4 3 2 1 c. Time out/ 

Isolation/ 
Separation 

d. Criticizes/ 

Justifies authority 

4 3 2 1 e. Conference 

4 3 2 1 f. Punishes 

^321 g. Problem solving 

5 4 3 2 1 h. Ignore 



7 6 
7 6 
7 6 



7 6 5 4 3 2 1 



5 
5 
5 



6. Interacting 

* ' Effectively 

5 4 3 2 1 a. Listening skills 

5 4 3 2 1 b. Expresses feelings 
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TO: Observers in the Classroom Organization Study 
FROM: Linda - ' . 

DATE: August 31, 1977 

Procedures for completi ng and tuming in your narrative, follow-up questions. - 
student e ngagement ratings, and Emmer teacher ratings 

- Now that we have had some time to try our original procedures for completing 
all of the forms and ha%'e discovered the problems, we have formulated a new set 
of procedures which should insure maximum information turned in with minimum time 
spent by the observers. At least, I hope so. So, this here memo is to say in 
writing the points I made yesterday in conversations with you. These instructions 
replace all previous ones . 

At the end of an observation, you are responsible for turning in four 
types of data: 1. the narrative 

2. the follow-up questions 

3. the student engagement ratings • • 

4. the Emmer teacher ratings (Component Ratings) 

In the Pit, you will find a set of four files for each teacher, one file 
for each kind of~^ data. (Isn't that clever?) Put completed observations here. 
CHECK TO SEE THAT ID INFORMATION IS ACCURATE! ! ! WOE BE UNTO HIM/HER WHO DOESN'T 
HAVE ALL NECESSARY ID INFO DOWN AND PAGES NUMBERED! ! ! See below for special 
note about ID info on the follow-up questions. 

Now, for the finer details: 

1. The Narrative . You have several options for finalizing your narrative, 
depending on what is easiest for you. The final objective is to have all 
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Important Information about the observation accessible to a reader. If 



you can turn in your rough notes with minor editing, that's great. If you 
prefer to rewrite, that's great, too, although it will take you longer, as 
most of us have already discovered. If you feel the need to rewrite and/or 
expand your notes, please consider dictating them. This is a good way to get 
all information down in a very short time. 

If you decide to dictate , follow these procedures: 

1. Begin each tape (or each side of the tape, if you're long-winded), 
with the ID information and the side number and tape number (e.g., "This is 

side 1 of tape 1 of the narrative for Ms. at school, 

August 30 in a.m., Anderson as observer, 22 children present in class.") 

2. Affer recording a few minutes, stop and check to make sure you are 
being picked up on the tape. (You haven't really experienced frustration until 
you h^ve blabbered on for 30 minutes and discovered that you weren't recording.) 

3. Whenever you have a time to report (i.e., any of the information in 
the column at the left of the narrative forms), say so on the tape, so that this 
will be typed in at the appropriate place' in the narrative. For example, 

"T. circulated around room, talking to those students whose hands were raised. 
BEGIN TRANSITION: 10:47. T. went to front of ^ room and said, 'It's time to 
clean up . . . . .STUDENT ENGAGEMENT RATING 11.'" Also note in this way the 
times at the tops of new pages by saying, for example, "TIME IS 11:15" at the 
appropriate place. 

4. When you finish your narrative, say "End of dictation," and check again 
to make sure you were recording in the last few minuses. (You don't need to 
listen to the entire tape.) 

5. LABEL THE TAPE CLEARLY WITH THE NAME OF THE TEACHER, SCHOOL, DATE, AM or 
PM, AND YOUR NAME.. 

erIc 
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6. Since you are not yet turning in the narrative to be filed, put a page 
in the teacher's folder that says that you have dictated the narrative. 

2- Follow-up Questions. For every observation, you should also fill in a 
set of follow-up questions. Note the ID info on the first page, and number all 
pages throughout. Be sure the set is completely stapled together. Someone with 
more time later will finish the ID headings on all pages after the first, but 
you must number the pages. - 

Everyone should addre ss every question . The three options for .responding 

are: 

1. If you iiave addressed the question in your narrative in a way that 
would be clear to a reader, then initial the question. (Do not cross reference 
by page and line num^r as we originally said.) This' is easier to do, but 
please take it seriously. ' s : ' < 

2. I£ the question is not answered in your narrative because it is not 
applicable, then note "n/a.'' 

3. If the question is not answered in your narrative but could be answered 
about the observation, or if you feel that it is necessary to expand on the 
information in the narrative to completely address the question, then write 

in the additional answer in the space provided. If the space is not adequate, ' 
then use the additional blank sheets with the ID field, blanks at the . top. ■ 
If you do use an extra page, then PUT THE NUMBER OF THE QUESTION AT THE TOP and 
insert it into tha follow-up question set after the page on which the question 
was asked. (Number the inserted page in sequence.)" 

If you are dictating your' narrative , and would also like to dictate your 
follow-up questions immediately afterwards, then please do so. As you dictate, 
follow the same procedures given for dictating the narrative, making it obvious 
to the typist that you are switching to the follow-up questions. He or she will 
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then move to a blank form, type in the ID info which you will give, and proceed 
to type in what you say for each question. You can summarize responses to 
clusters of questions (e.g., "Questions 64-67 are not applicable, 68-76 are all 
addressed in the narrative.") 

If you dictate the follow-up questions, fill out the chart and file it 
along with a note saying that you have dictated the rest of the follow-up 
questions for that observation. 

r 

^* Student En^aRement Ratings , Complete the information on this page and 
turn it in as is. 

^- Emmer Teacher Ratings or Component Ratings , whichever we are calling 
them. NOTE: Add to each form you use a series of numbers beside the heading 
"Disru^.tl' student behavior" and rate the frequency of this as described in 
the manual. 

Reminder: There is an observer meeting at 2:30 on Thursday, Sept. 1. 
Please bring your questions and problems, and we'll work out any other wrinkles. 
By the end of the three weeks everything should be completely settled!.!!!!!!!!! 




I 
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OBSERVER- INTERVIEW - END OF FIRST THREE WEEKS 

Questions to.be asked observers about each of their- teachers ! 

K What was the basic organizational climate of this classroom? 
(jn terms of effectiveness—chaos vs, mild ripples) 

2. Do you think that the climate has gotten more or less organ i zecK' and 
smooth over time? 

3. If the class has become less organized than might be desired, what do 
you think could be done now to recoup? 

4. Did the teacher have credibility with students? What kinds of evidence 
are you considering in your answer? 

5. How consistent was he or she in following through on directions and/or 
corrections? 

6. What consequences to misbehavior were evident to the students? 

7. How. strongly did the teacher feel about teaching the basic skills- 
reading and associated language arts, writing, math? What evidence 
are you cons i der ing. i n your answer? 

B. How hard were the ki ds " p_u<shed to perform? What was the pay-off for 
the student for performing well on academic tasks? 

9. What seemed to be the most important thing to the teacher? 



Overall questions to observers ' 

1. What five suggestions would you make to a new teacher about how to 
start the school year? 

2. What five things would'you tell a new teacher to avoid doing during 
- the first three weeks of the school year? 
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Teacher Interviews and Questionnaires 

The observations provided a wealth of information about the 27 

• r 

classrooms, but much important knowledge can not be obtained through 
observation, especially information about planning, decision-making, or 
constraints affecting the teacher. Therefore, each teacher completed a 
questionnaire and was interviewed twice, early in the school year. 

Interviews . The first interview' s- purpose was to gather 
information about aspects of classroom organization which were not 
observable. Several specific questions were also asked about the 
organization of reading instruction. This interview was conducted four 
to six weeks after the beginning of the school year. The questions 
included in this interview . follow this section. 

The second interview's purpose was to determine each teacher's 
Level of Use (LoU) of overall c lassroom organizat ion and the teaching of 
reading. The second interview was conducted by members of the staff of 
the PAEI/CBAM program at, the R&D Center during ^November according to 
interview techniques!^ developed by them. The LoU reports the behaviors 
described by "users" of an innovation, and has been used extensively to^^ 
diagnose the status of the users and-^their relationship to the 
implementation of an educational innovation. .The LoU provides a score 
for each seven aspects of use of an innovation in addition to an overall 
rating. Thes^ categories are listed on the rating sheet which follows. 
:The information was collected in the -Classroom Organization Study to 
determine whether the LoU scores could be. used to^ predict management 
effectiveness. . , ^. 

Although classroom organization had never been considered an 
innovation per se , it was felt that it could be studied by similar 
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techniques since it can be viewed as something which must be learned 
through use. Therefore, an important research question in this study 
was how easily the concept of Level of Usa and the associated 
tnathodology could be applied to a more global concept like classroom 
organization. 

Questionnaire. The Stages of Concern questionnaire was also 
completed by the teachers in the month following the first interview, 
once for ov rail clasSroom organization and again for reading 
instruction. These were also based on work done by the PAEl/CBAM staff 
at the Center, and will provide information about teachers' most intense 
concerns about classroom organization and reading at the time of the 
interview. Copies of the two questionnaires follow. 



11-55 



Classroom Organization Study — Teacher Interview ffl 

The following questions were asked to the teachers during their first 
Interview. The order of questions was not necessarily followed, if the 
teachers provided needed information in an answer to a previous question 
or spontaneously answered a question without being asked directly. The 
Interviewer was also free to follow up on answers with other probes that^ 
seemed important at the time. However, this set of questions served as 
a general guideline for the interviewer. 

An Introduction preceding the Interview emphasized the purposes of the 
interview: 

o 

1. to Identify constraints 

2. to gather information on topics which could not be observed, 
such as planning activities 

3. to learn of teacher's reasons for various strategies which we 
observed. 

The teachers were then told that the interview would concern both 
classroom organization in general, as well as organization of reading as a 
specific area. We are i nterested in other subject areas too, we said, but 
would only discuss reading at that time. 

I. General classroom organ i 2:at ion and structure 

~We are interested in your overall classroom organiation, both in 
your descrip* of it and in how you developed it. First, would you 
teil me abcat what kinds of things you did before the first day of 
classes to g^^t ready for school? 

PROBE FOR If NOT MENTIONED IN ANSWER TO ABOVE: What did you do to get 
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to get your room ready? What about lesson planning that went on before 
classes started? What meetings and in-services were you involved in dur 
ing the weeV-before school started? 

—Thinking back to the first few weeks of school, what specific things 
did you want to accomplish with the class in terms of their general be- 
havior? 



0 



PROBE IF NOT ANSWERED ABOVE: What did you do to accomplish these things 
—What sorts of rules do you have in your classroom about what students 
can and can't do? Were any posted or otherwise written down? What are 
kids forbidden to do in the classroom? Are there procedures for what 
kids can do on their own: If not specific, probe further regarding 
bathroom, water, pencil sharpening, talking, activity. 
— How did you introduce these rules to the kids? 

—What happens to srudents when they break a rule or otherwise do not 
perform as des i red? 

PROBE IF NOT ANSWERED ABOVE: What happens when a student does not fin- 
ish and/or turn in his or her daily work? What do you do? 
—What other rules are you expected to support that are school or 
district pol icies? 

PROBE: Were district guidelines given for consequences? 

—Besides rules, what other things in your class encourage students to 

behave well? How do you deep them working?' 

—What problems and constraints have you encountered that have gotten' 
in the way of your classroom organization? What things make it more 
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difficult to run your class as you would like? 

PRoBE IF NOT ANSWERED ABOVE: Are there constraints in the area of 
scheduling imposed by vrhool? Scheduling of individual students in 
activities outside of the room, like speech. Title I reading, etc.? Pro- 
blems with supply of materials? Paperwork not directly related to in- 
struction? Required testing? Physical constraints (building or room)? 
—How do you deal w (h these constraints to minimize their effect? 
(Interviewer may have to choose specific ones to focus on.) 
—What techniques have helped you- run your class more efficiently? 

Reading instruction 

—Turning now to specific subject master, would you briefly describe 
your daily routine for teaching reading? 

PROBE IF NOT ANSWERED ABOVE: Where do the kids go for readi ng-during 

the morning? What do you do? Are they working in groups or individually? 

— What materials do you typically use to teach reading? 

PROBE IF NOT ANSWERED ABOVE: How did you get these materials (i.e., 

checked out by school as matter of course, left from last year, scrounged 

before school started, etc.)? 

—How dM you introduce your system of teaching reading to the students? 
PROBE IF NOT ANSWERED ABOVE: When did you start it? How were the pro- 
cedures for movement around the room, getting materials, contacting the 
teacher, etc., explained to the students? 

—What made you choose this particular system of teaching reading? 
—When you do your daily planning for reading, what kind of things do 
you th i nk of ? 

PROBE IF NOT ANSWERED ABOVE: How do you consider differing achievemenr 
levels? How do you think about materials to be used? (i.e., obtaining, 

« ^ 
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arranging) 

--What constraints have been imposed on your reading program this year? 
— What has been helpful in your reading program? 

PROBE-IF NOT ANSWERED ABOVE: Ask about scheduling of students for other 
places in school, scheduling of whole class, access to ,7,aterials, pro- 
blems with teaming, .problems with grouping, testing, school- policies, 
paperwork, etc.) 

-How satisfied with your reading program are you. in terms of your over- 
all objectives? 

PROBE IF NOT ANSWERED ABOVE: What would you like to be doing differently 
or have happen differently? 

-What do you think you will be doing with'your students in reading by 
the end of the semester? 

PROBE IF NOT ANSERED ABOVE: Push for planning, long-range game plan, 
plans for adapting program, whether this teacher selects cues from stu- 
dents to guide the pace or whether she sets an arbitrary end point, 
(i.e., finish half of the basal by Christmas). 

-For either reading or other subject matter areas, how do you use 
other people, such as aides, parent volunteers? 

-Do you have a student teacher? How does that affect your organization 
and p lanni ng? 

"Looking back over your experiences at the beginning .bf school, what 
suggestions would you make to other teacher^ about how to best begin 
the year? We're interested in both overall organization as well as 
beginning reading instruction. ' ■ • 

-Anything else you wanted to say that I haven't asked you about? 
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SAMPLE FORM: LEVEL OF USE RATING SHEET 



Tape {>: 
Date : 



/ 



Site: 
> I.D. //: 



Interviewer ; 
Rater : 



Lev<^ 1 



Non'-Use 
U . 'P. A 

0<rlentat ion 
D.P. B 

Preparation " 
D.{\ C 

Mechanical Use 
D.P. D-1 

Routine 
D.P. D-2 

Refinement 
D.P. E 

Integration 
D.P. F 

Renewal 



Knowledge ^"^'^'i'on ^'^''''^^"8 



^\ssessing Planning 



Status -n c J 

Reporting Performing 



0 
I 

II 
III 
IVA 
IVB 

V 
VI 



I 

II 
III 

IVA 
IVB 
V 
VI 



I 

II 
III 

IVA 
IVB 
V 
VI 



I 

II 
III 
IVA 
IVB 

V 
VI 



I 

II 
III 
IVA 
IVB 

V 

VI 



I 

II 

III 

IVA 
IVB 
V 

VI 



I 

II 
III 
IVA 
IVB 

V 
VI 



0 
I 

II 
III 
IVA 
IVB 

V 
VI 



User is 
not doing: 

No Information 
in interview: 



ND 
NI 



ND 
NI 



ND 
NI 



ND 
NI 



ND 
NI 



ND 
NI 



ND 
NI 



ND 
NI 



Past User 



Estimated past LoU 



The Amount of information in the interview was: 
The interviewee: 



^interviewee: 



insufficient 
for rating 

does not fit 
on the chart 

was very difficult 
to interview 



5. 



6 

9 



very adequate 
for rating 

fits well 
on the chart 

was no prbbli 
to interview 



SAMPLE FORM: 



SoCC A3 
AISD Reading 



Concerns Questionnaire 

Unique Number 
School 



Date Completed 



Important for continuity in processing this data that we have 
a unique number that you can remember. Please use: - 

Last 4 digits SSif 



DIRECTIONS: This questionnaire is designed to explore what teachers are 
nTllT. different points in their, careers. There are/'of course, 

no right or t/rong answers: each person has his or her own concerns. 

wSat'Jhev think%h'v TTA''° '"'"''^ q-stions like these in terms of 
about ?n^^ ? ^^°"^d concerued about or expect to be concerned 
? u ^^^^ what is wanted here. We would like to 
know only what you are actually co ncerned about MOIT in terms of your 
teaching of reading. — yyj^t. 

This questionnaire is used for several purposes. Therefore, a good part of 

? °" Qu esti on n ai r e .ay appear to be of little rel .L.JT 

completely irre levant to you at this time. For the completely irrelevan t 
Items, please circle '-O" on the scale. Other items wiU represent thise ' 
risS:r":n"the^eat!' ^"^^"^ ^-""^ °^ ^ l^oulT.Tlll... ^ 

For example : , 

0 1 2 3 A 5 6(7) This statement is very true of ne at this time. 
0 12 3 (T) 5 6 7 This statement is somewhat true of me now. 
0 0 2 3 5 6 7 This statement is not at all true of me at this time. 
® ^ ^ 3^ ^ ^ 7 This statement seems irrelevant to me. 
Please^respond to the items in terms of your present CONCEWIS ABOUT TEACHIIIG 

Thank you for taking time to complete this task. 
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0 1 2 3 A 5 6 7 

Not true of me now Somewhat true of me now Very true of me now 



01234567 



0 1 2 3 4 5 6 7 



I am concerned about students' attitudes tov;ard ny teaching of 
reading. 

I tiow'know of some other approaches that might work better. 



01234567 I don't even know about the teaching of reading, 



0 1 2 3 4 5 6 7 

01234567 

0 1 23 4567 
012 3 4567 



I am concerned about not having enough time to organize myself 
each day. 

I would like to help other faculty in their teaching of 
reading. 

I have a very limited knov/ledge about the teaching of reading. 

I would like to Icnow the effect of reorganization on my 
professional status. 



01234567 I am concerned about conflict between my interests and my 
. responsibilities. 

01234567 % am concerned about revising my teaching of reading. 

I would like to develop working relationships with both our 
faculty and outside faculty v/ho are teaching reading. 

I am concerned about how my teaching of reading affects 
students. 



0 1234567 
0 1 2 3 4 5 6 7 



0123456 7 I am not concerned about my teaching of reading. 

01234567 I would like to know who will make the decisions in the new 

system. - 

01234567 I would like to discuss the possibility of teaching reading. 

0 12 3 4.567 I would like to know what resources are available If I decide 

to change to another way of teaching reading. 

01234567 I am concerned about my inability to manage all that teaching 

of reading requires. 

01234567 I would like- to know how my teaching or administration is 

supposed to change. j j 

0 1 2 3 A 5 6 7 I would like to familiarize other departments or persons with 

- the progress of my teaching of reading. 

Copyright, 1974 
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3 4 5 6 7 



0 1 2 

Nos true of me nov Somewhat true of me now Very true of me no,^ 



012 3A567 
0 123A567 
0123A567 
0123A567 

0 12 3 4 5 6 7 

0 1 2 3 A 5 6 7 

01234567 

01234567 

0123 4 567 

01234567 

0 1 2 3 4 5 6 7 
012 345 6 7 

01 2 34567 

0 1 2 3 4 5 6 7 

0 1 2 3 4 5 6 7 

0 1 2 3 4 5 6 7 
0 1 2 3 4 5 6 7 



I am concerned about evaluating my impact on students. 

I would like to revise my teaching of reading. 

I am completely occupied with other things. 

I would like to modify my teaching of reading based on the 
experiences of pur students. 

Although I don't know about the teaching of reading, I am 
concerned about things in the area. 

I would like to excite my students about their part in mv 
approach to reading. 

I am concerned about time'- spent working with nonacademic 
problems related to the teaching of reading. 

I would like to know what the teaching of reading will require 
in the immediate future.- 

I would like-to coordinate my effort with others to maximize 
the effects of teaching reading. 

X would like to have more informa';ion on time and energy 
commitments required by the teaching of reading. 

I would like to know what other faculty are doing in this area. 

At this time, I am not interested in learning about the teach- 
ing of reading. 

I would like to determine how to supplement, enhance, or 
replace the way I teach reading. 

I would like to use feedback from students, to change W read- 
ing program. 

.1 would like to know how my role will change when I am teach- 
ing reading. 

Coordination of tasks and people is taking too much of my time. 

Ltt^r^h^^^ r '"^7 '•'^y" °^ teaching reading are 

better than what I have now. 



_ ' Copyright, 1974 
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The following questions are virtually identical to those you just 
completed on reading. For these, however, we v/ould like for you 
to indicate your present concerns about classroom orf^anization . 
By classroom organization, we mean the way you organize and manage 
instructional materials, instructional time, . contacts V7ith indi- 
vidual pupils, and the many activities that 50 on during a school 
day. 
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' • 2 3 A 5 6-7 

Not true of ne now Somewhat true of ne now Very true of ne now 

Jr5anL°a"on!'^ students' attitudes toward my classroom 

I now know of some other approaches that night work better. 
I don't .eyen know what ray classroom organization is., 

each day!""''^ ^^''""^ ^° organize myself 

I would like to help other " faculty with their classroom 
organization. 

I have- a very limited knowledge about classroom organization. 

I would like to know the effect of reorganization on my 
professional status. . ^ 

I am concerned about conflict between my interests and my. 
responsibilities. ^ 

I am concerned about revising ny classroom organization. 

I would like to develop working relationships with both our 

Innl nl ""f ^^culty in Order to optimize' their class- 
room organization. 

I an concerned about how my classroom organization affects 
students. 

I am not concerned about classroom organization. ' 

I would like to know who will make the decisions in rhe new 
system. " 

I would like. to discuss the possibility of using classroom 
organization. 

I would like to know what resources are available if X decide ' 
to use classroom organization. 

I an concerned about my inability to manage all that my class- 
room organization requires. ' i-ucit my class 

iu^'^n'ed'in^ i° °^ '^^'^'''"S or administration is 

supposed to change. • 

I would like to- familiarize other departments or persons witS 
my classroom organization. ^ ^^^^ 

Copyright, 1974 



0 1 2 3 4 5 6 7 

01234 5 67 
0 1234567 
01234567 

0 1 2 3 4 5 6 7 

0 1 2 3 4 5 6 7 
0 1 2 3 4 5 6 7 

0 1 2 3 4 5 6 7 

0123 4567 
0 12 3 45 6 7 

0 1 2 3 4 5 6 7 

0 1 2 ^ 3 4 5 6 7 
0123456 7 

0 /1 2 3 4 5 6 7" 

0 1 2 3 4 5 6 7 

0 1 2 3 4 5 6 7 

0 1 2 3 4 5 6 7 

0 1 2 3 4 5 6 7 
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0 1 

Not true of me now 



3 A 

Somewhat true of me now 



Page 6 

6 - 7 
Very true of me now 



0123A567 
0 123A567 
0 1 2 3 A 5 6 7 
0 1-234567 

0 1 ,2 3 A 5 6 7 

0 1 2 3 A 5 6 7 

012 3 A567 

0123A567 

0123A567 

0 1 2 3 A 5 6 7 

0 12 3 A 5 6 7 
0123A56 7 

dl2 3 A5G7 

0 1 2 3 A 5 6 7 

0123A567 

0 12 3 4 5 6/ 
0 1 2 3 A 5 6 7 



I am concerned about evaluating my impact on students. 

I would like to revise my approach to classrooiii organization. 



I am. completely occupied with other things. ' 



I would like to. modify my classroom organization based on the 
experiences of our students. 

Although I don't know about classroom organization^ I am' con- 
cerned about things in the area. 

I would like to excite my students about their part in my 
classroom organisation. 

I am c.;ncemed abcut time spent working with nonacademic 
problems related to classroom organization. 

I would like to know what the use of my classroom organization 
will require in the immediate future. ^ 

I would like 'to coordinate my effort with others to maximize 
the effects of classroom organization., 

I would like to have m6re information on time and energy 
commitments required by my classroom organization. 

I would like to know v;hat-qther faculty are doing in this area 

At this time, I am not interested in learning about classroom 
organization. 

I would like to determine how to supplement, enhance, or 
replace uy classroom organization. 

I would like to use feedback from students to change my class- 
room organization. 

I would like to know how my role v;ill change when I am using 
classroom organization. 

Coordination of tasks and people is taking too much of my time 

I would like to know how other v/ays of classroom organization 
are better than what I do now. 



Copyright, 1974 
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Thank you for responding to the two questionnaires. Bef....r. vor. stop 
please take a minute ta provide the foljowin;; infornatioa. 

Age (circle one): • 

26 --29 30 ^ 39 40 - 49 over '49 
Training 

Degrees earned 



Date of last degree 

College awarding 
last degree 



Experience 

llhat grade level(s) are you now teaching? 
Total years of teaching" experience 

Years experience in this school 

Years experience in AISD / 



How many years have you taught your current grade? 
Which grade levels have you taught? \ 



Role 

Are you a member of a tea^ 
Are you a team leader? 



7 



(circle, one) 
Yes No 

Yes Wo 



Are you teaching in a self-contained classroom? Yes 



No 



Co 
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DATA CX)LLECTION AFTER THE FIRST THREE WEEKS 
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SECTION III 
DATA COLLECTION AFTER THE FIRST THREE WEEKS 

Classroom Observations After November 
After the first three weeks of school, observations were 
interrupted for about six weeks. During this time, the data from the 
first three weeks were transcribed and organized, and some modifications 
to the instruments were made. Observations were then resumed in 
November and continued through May, in order to gather data about year 
long management and organization trends and effects. In late March, one 
teacher quit teaching and another transferred to another position. Data 
from these two teachers* classes were used in subsequent analyses, 
because by the time they left, a sufficient number of observations had 
been made to provide an accurate assessment of their management 
behaviors. 

The teachers were observed about once every three weeks for a total 
of eight or nine times, yielding an additional eighteen to twenty hours 
of observations per teacher. Each teacher was seen by two observers who 
alternated visits. Morning observations were approximately two and one 
half hours long and afternoon observations lasted about two hours. Four 
regular observers were used and the principal investigators also 
observed occasionally. 

The 'observations were similar to those conducted in the first three 
weeks, with some minor changes in the instrumentation. The folfow-up 
questions were no longer used, and instead there was a more detailed set 
of guidelines for the narrative record. Categories in the Student 
Engagement Ratings were subdivided and the component ratings were no 



longer used on a regular basis, althouji^h periodic ratings of the 
teachers were done. A time log was added, to provide data on time use 
in various content areas .and instructional groupings. Therefore, each 
end-of-school observation consisted of a narrative, a set of Student 
Engagement Ratings, and a time log. 

1. Narrative . The questions to be addressed in the narrative were 
somewhat different from those which were of concern at the beginning of 
school. The remainder-of-year focus was on maintenance of organization 
and modifications throughout the year- The manual for the narratives 
f o 1 lows . 

The most important addition in eitiphasis was to focus as much as 

possib le on reasons that students were of f- task. That is, observers 

were asked to include information about antecedent or simultaneous 

conditions that could be linked to individual students or sub-groups of 

students who were not engaged. In this vay, the student engagement 

ratings v/ere linked to classroom events, - providing data for inferences 

about relatio.iships between teacher organizing techniques and student 

on-task behavior. For example, these two samples sugge-st^^linkages 

between teacher actions and student behaviors. \ 

"The boy by the bathroom doors looks up every t im(^ someone 
passes him, and he seems to be a kid who is very easily 
distracted. It's taken him 30 minutes to copy his ten ' 
spelling words, but the teacher hasn't realized this yet." 

"Ihe teacher's explanation begi^n by np.aking sure everyone 
was watching her and silent. She then explained Part 1 and 
let them begin it. When almost everyone had finished, she 
cal led for their at tent ion again and present;.ed direct ions for 
Part 2 and after-work options. This interruption did not 
seem to bother the children, since all of them got back to 
work right away , and soon seemed to 

be looking at Part 2 in their books. Since t : instructions 
for 2 were so different from 1, I think the 1 ids would have 
been confused if they had gotten all the directions at once. 
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The teacher spent the rest of the time working with three 
students at the reading table. There was very little off-task 
behavior in the students at their seats." 

2. Time Log . After the observation was completed, the observers 
filled out a time log which included information on how long 
transitions took, how ir./ch time was spent on each type of activity, and 
how many stud^ts were involved. The time log can be used along with 
the narrative to get a detailed chronology of classroom activities 
Instructions for the time log and a sample form follow. 

^- Student Engagement Ratings , As was done in the first three 
weeks, every 15 minutes the observer counted the number of students who 
could be classified in eaclf category of student engagement. However, 
there were some slight diange/ in th^ cstegori<is. In addition to 
counting studv-nus, the observer noted rhe ::Dde number for a set of 
categories for lesson fomat and saLjp.ct itij.V.ter. Other changes 
included di riding the academic and procedural on-task categories into 
"definitely on-task'' n-d ''probably oo-Ka^c'' for each type of activity. 
A category wa.^ added for ^^de.^d tfnie/* in which a student has nothing 
specific to do, and the meaning of the "Can't teil'^ category wa^ 
limited to studtnts who were rot . ibJ.t- tc the observer. Ihe manual 
for these revisions of the S/:udeint Ev.ga^^.-zn'^nt: ^/latings Z^.liows. 
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MAN1..VU FOR NARRATIVES 
TAKEN IN THE SECOND PHASE OF OBSERVATIONS 
IN THE CLASSROOM ORGANIZATION STUDY 

The guidi-ff framework for this study has been, from its beginning exa- 
mining four aspects of classroom organization: the management of time, materials, 
contacts with students, and constraints imposed on teachers. The four topics 
remain critical to the study, but experience during the first few weeks of 
school has led us to reorganize the questions that we are asking, in order to 
more specifically tap important aspects of cflassroom organization. Therefore, 
the following description does not replace earlier conceptualization of this 
study, but instead refines the questions based on our experience in classrooms. 

While in the classroom, the observer- is responsible for three activities. 
The most important is to be taking notes in the form of a narrative record 
which addresses the questions discussed below under "Narrative." In the process 
of doing this, the observer will also note any times that are necessary to 
complete a t?"^^' log after the observation. * Criteria for such times are also 
discussed bel^w under "Notations of tirne." The third responsibility of the 
observer is to complete a Student Engagement Rating every fifteen minutes, 
by noting the number of students falling into each o'f eight categories of 
engagement, and by noting certain information about the context of the classroom 
at that time. Definitions of these categories are noted below under '^Studen^ 
engagement ratings." 
Notations of Time 

The observer should note the time at the beginning and end of an observation, 
* and should include in the narrative record the times at which the subject matter, 
the major activity, content, and/or format are changed within the classroom. 
This would include the beginning and ending of all transitions, either between 
or within subject matters. The observer . should also note the time of any 
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interruptio:\. In general, therefore, the observer is being asked to note any 
times which signal a change in the classroorii organization. IVhenever an observer 
cannot decide if a time should be noted or not, she should go ahead and note it. 
It is better to have too many times than too few. 

After the observation is completed, the observer should fill out the 
time log on the forms provided. This would include completing the identification 
field at the top of each page, and describing each interval of time in terms of 
the Start Time, Stop Time, number of students involved, and a brief description 
of the activity. The time log has space for three concurrent activities to 
be noted, so that it will be possible to describe the different schedules of 
various groups in the classroom. This time log should include intervals spent 
in transition, as well as intervals of time spent in actual instruction and work 
on assignments. When it is completed, it should be possible to follow the 
schedule throughout the day for all of the students in the cl<iss. In: ^ ioes 
not mean that it should be done in terms of individual students, b^yt x^.ther in 
numbers of students in each subgroup who are following diffeii-' ;chedules. 
An example of a completed time log is attarhr-^a. 

If the observer has more than three ti . tl^i; :o note because four or more 
groups are following different schedules, ihe d\onld os<* an extra page, rather 
than trying to squeeze all four on to the three sections of one page. Wien completed, 
the pages should be numbered consecutively and noted a^E »Tage of 
Narrative 



The narrative which is__compXel-exi-j.n-the-^coM^^ of observation will 
follow a general outline which lists the important questions to ask duffing 
each observation. This is a change from the very specific questions which were 
asked of observers with the 62 follow-up questions of the first three weeks' 
observation. The outline is provided below with some discussion of each of the 
questions and examples of important activities. 
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The outline is divided into two major sections which each focus on either 
teacher or student activities. There are more specific questions asked about the 
teacher's activities, but this should not lead the observer to think that these 
are more important than the students' activities. It is important to note the 
relationship between what the teacher does and what the student does; that is, the 
reaction of one to the other should be a very important aspect of the narratives. 
In fact, probably the most important question of the entire narrative is the last 
one included on this outline, "IVhy are students, either as a grv&p or individually, 
off or on-task?" However, in order to answer this completely, it is important to 
have the information provided by answers to all of the other questions. The outline 
below provides discussion of each question. 

I. Teacher Activities OVhat are the activities engaged in by the teacher to establish ' 
■ and, maintain the classroom organization that day?) In general, the observer will 
be noting both explicit organizing activities and move5 by the teacher, as well 
as i mplicit ones. Implicit organization would inclua • procedures and routines 
that were established previously and which are used o- a regular basis. As the 
observers become more familiar with the classrooms, and know that certain procedures 
have been described in detail, they can refer to them more briefly as time goes on. 

A. Preparation (l^at has the teacher done to get ready for that day?) The 

observer should note both explicit and implicit planning and pr;^paration which 
are evident^ Likewise, it should be noted if there has been a lack of plan- 
ning. Planning activities includes preparation and organization of materials 
and supplies. It will also include whether or not the teacher has a clear 
idea of the sequence of questions or sequence of information presented to 

the students.- IVhenever the observer feels that a. problem has arisen due to.. 
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the lack of planning or preparation in either of these areas, she should 
feel free to note her interpretation of the situation. 
I^^cher communication to students. (JVhat does the teacher communicate 
to the students about what they are supposed to be doing?) To answer 
this question, the observer should note the instructions, directions, 
e.xplanations and corrections delivered to the students about their 
activities and use of materials. These communications may be explicit 
and given completely within a single observation, or they may be implicit 
as part of a procedure of routine which the students cbviously know. 
Observer judgement will be involved here in deciding how many details 
to include about the teacher's verbal instructions. In some cases, it 
will be necessary to repeat them almost verbatim. In other cases, they 
can be described and swmarized, with regard to the amount of detail 
or clarity involved. The observer should keep in mind that persons 
reading. the narrative may want to trace later student on-task or off -task 
behavior to the information given to those students before beginning a 
task. Again, as the observer becomes more familiar with the procedures 
and routines in a classroom and particular teachers' styles, it may 
be possible to include less detail on such matters. 

Another impoitant component of teacher communication to students 
is the extent to which and the way in which teachers use praise and 
criticism. It has become increasingly clear to us over the last few 
years that the use of praise and criticism is a complex matter. Some 
teachers are very effective in motivating students to perform well-^-hrough 
their U5.., while other teachers use so much that they make_ the situation 
worse. One type of praise which many tetfchers use with varying degrees 
of success is public praise to emphasize a positive behavior-"I like 
the way that Susie is sitting quietly and listening to me." Likewis^e. 
a little bit- of criticism is sometimes highly effective with some students, 
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but can be misused easily. The observer should remain aware of the 
difficulty in using praise and criticism effectively, and should note 
instances in which they are used well and misused. Such descriptions 
should include, when evident, the antecedents and consequences of the 
use of praise and criticism. Again, observers should feel free to make 
any interpretive comments regarding this. It may be useful to the 
observer to think of activities in the classroom in terms of these 
three categories: 

i. Academic activities and use of materials in them. The narrative should 
include descriptions of the actual assignments given to the students in 
the subject matter areas, such as written seatwork, reading, building a 
project for art or science,- or completing math problems out of the text- 
book. 

ii. Procedural activities and use of materials in these. This includes 
directions from the teacher about classroom: procedures which are not 
directly related to completing an academic assignment, such as transitions 
within the room or into and out of the room, getting materials from a 
rrhelf and returning them, and going to the bathroom, getting water, or 
going to the pencil sharpener. There will probably be 'fewer of these 
communications than there were in the first three weeks of school, although 
some can be expected throughout the year. It will be especially important 
to note "the changes in a standard procedure or Schedule and the way that 
the teacher explains this to the students. For example, throughout the 
year, a teacher may be introducing new .learning centers 'and materials to 
the students. It would be very important to note the specific direct ions 
and explanations given to the students then, 
iii. Behavioral corrections about unsanctioned activities or *unsanct ioned use 
of materials.' In this category would be. included teacher corrections to. 
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studen^ who have broken some classroom rule or who are misbehaving. It 
iould also include punishment that was threatened or delivered by the 
. leacher. The observer would also want to note times when threats were 
not followed through on. and also times when students were obviously 
misbehaving but were not corrected by the teacher. In this case, there 
has been -a lack of direct communication from the teacher which is as 
important to know about as the presence of some communication. In general, 
this statement applies to all of the questions asked under the heading of 
"Teacher communication to students.-" Again, the observer should feel 
free to note any interpretations or suggestions about why problems arose 
due to Jack of clear communication to students about what was expected. • 
Teacher contacts with student s and monitoring of c1.s..nnn, (How does the 
teacher know.what is. going on with each student? How does the teacher moni- 
tor the class? How does the_ teacher allocate contacts between the students 
and him/herself?) ivhile the focus of Question A above was on activities' 
the teacher does alone without the students, and the focus of Question B was 
on communication from the teacher to the students, the focus of this set of 
questions is on the teacher receiving some kind of communication or feedback 
from the students about how well they are working or how much they understand. 
One important aspect of classroom organization is how the teacher sets this 
up so that it is possible for him or her tO: maintain as much contact as is 
necessary with the students in order to get feedback from them about their 
understanding, and to give feedback to the students about their work and 
behavior. Again.' it may be useful to the observer to think of the classroom 
in terms of three major types of contacts: 

i. Academic contacts. It is imoortant for the observer to note how the 

teacher selects students to answer in a public setting, such as a question- 
and-answer discussion. Some teachers rely primarily on use of volunteers. 
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while some teachers allow student call-outs. In both of these cases, 
the students are often more in control of «who deals with the teacher in 
the public setting than the teacher is. Other teachers maintain more of 
this control for themselves by calling on students who have not voliUi- 
teered, or by using some sort of systematic way to call on students, such 
as going around the room in a pattern. The observer should note which- 
ever techniques are used by the teacher to select students in public 
quest ion-and-answer settings. 

It is also important to note how the teacher maintains contact with 
students who are doing private work, such as seatwork or working at a 
learning center without being under the teacher's direct control 'at the 

time. Some teachers only deal with students who approach them or raise 

■J 

their hands, while other teachers very systematically circulate throughout 
the room, checking on students. The observer should note how the teacher 
distributes contacts in this '^ctti^g also. 

In addition, any information on how the teacher gives feedback to the 
students about mistakes made on their written work would be very valuable. 
(However, many times this is not evident.) A teacher ' .pends the first 
half-hour of every morning returning yesterday's work a. ' ^ling the 
students their mistakes is a teacher who is very deliberately giving the 
students feedback on their work. The actual curriculum content of any 
such contacts is not of great importance to the narrative, except when 
it seems to affect the students' attention to task. For example, if a 
teacher is circulating around the room and ngtices that a student has 
misunderstood how to do something and therefore she reviews the instruc- 
tions, this content should be noted. Likewise, if a student is making a 
lot of mistakes because he doesn't understand some concept and the teacher 
spends some time reviewing this, this should be noted, so t:V <t v. i: 
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distinguishable from problems arising from forgetting instructions about 
page numbers, how to work a puzzle, etc. 
ii. Procedural contacts. These would include the teachers' monitoring the 
students as they go through pre-established routines, procedures, and 
offering further explanation as necessary. An e.xajnple of a teacher 
actively monitoring such a situation would be one who stands at the front 
of the room and observes the students as they move through a transition 
to make sure that everyone is going where they should be going. An example 
of a teacher failing to do this would be one who turned her back to most 
of the class as they were moving through a transition. The actual content 
of such procedural monitoring and any contacts that result from them 
would be important to describe in order to clearly define any organiza- 
tional problems. 

iii. Behavioral contacts--^It is important to note how quickly the teacher 
responds to misbehavior by the students and when this is the result of 
systematic monitoring on his/her part. That is, what does the teacher do 
to make sure that he/she will spot misbehaviors? Likewise, what does the 
teacher fail to do that prevents him/her from spotting misbehaviors? 
Overall, the observer should feel free to note impressions about the ' • 
way monitoring systems or systems of allocating contacts seem to prevent, 
minimize, or cause problems. The observer .hould also note impressions 
^ about whether the teacher's contacts with the students consist primarily 
of academic, procedural, or behavioral contacts. Some tearhers never 
deal with students except when they misbehave. Other teachers have, things 
running so smoothly that they hardly over have behavioral or procedural 
contacts, and therefore most of their contacts are academic in nature. 
Such impressions are very valuable, especially when the observer can also 
provide reasons for them. . - 
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D. Constraints on classroom organization . (\^at gets in the way? i/liat 
interrupted the routine? IVhat constraints were evident and how d.^ the 
teacher deal with them?) The observer should note all interruptions of the 
classroom routine and try to classify them as internal or external to the 
class- That is, were they things that the teacher could have prevented or 
not? Likewise, any impressions should be noted about gen,eral conditions in 
the classroom or school that might make the teacher's job more difficult or 
which mean that he/she has to cope with a particular situation to minimize 
problems. An example of this might be having insufficient room for children 
to hang their coats but having students who must go outside several times 
. during the day to go to other rooms in the school. Other examples of con- 
straints are schedules imposed on the teacher for their entire class or 
groups within the class. IVhenever it seems pertinent, the observer should 
^ note any questions about whether or not the teacher has the power or authority 
to do certain things, or has access to resources that would solve the problem. 
For example, if problems are arising in the classroom because of students 
sharing materials, and it isn't clear to the observer whether or* not the 
teacher actually cuuld get enough materials, this question should be noted. 

Student Activities and Response to Teacher C^^^at studenc activities are expected 
and desired by the teacher and what activities are actually engaged in? That is.^ 
what is the focus of the students' attention/engagement?) This is probably the 
m-Jt critical question of the entire study . IVhenever the observer notices the 
probable relationship between student attention/engagement and something about 
the organization of the classroom, she should note that idea or interpretation. 
A. Counts .of students engaged in various categories will be noted on the Student 
Engagement Rating form. The definitions of these categories are described 
elsewhere in this manual. The observer should key each rating to the narrative 
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by noting the number of the rating on the narrative form, in the column 
"St. Ong.," at the line on the- page where the narrative was intei^-upted for 
the Student Engagement Rating. After taking, the rating, the observer should • 
try to describe in the narrative anything that accounts for students being 
on- or off-task. 

B. IVhenever it seems relevant to the observer, she .should note first names of students, 
This will result in informal case studies of cert.in students that stand out 

in the class. 

C. Probably students who are on- on off -task in a class will be that way for 
different reasons, depending on the student. IVhenever possible, the observer 
should note the apparent reasons for various subgroups of students. Some 
possible reasons that students may be off-task are: 

i. They don't have enough to do. and have finished the assigned work, or 
don't understand what they are supposed to do when they are finished. 
ii. The students have enough to do, but do not understand how to do it 
and can't get the teacher's attention and/or help, 
iii. The teacher is not monitoring the students and they start misbehaving 

and misbehavior continues. ^ 
iv. The students are distracted by something going on in the classroom. 
. V. The student has b^n working too long on the same thing and his/her 
attention span is short, 
vi. The student is bored with the assignment." 
vii.^The student is engaged in other activities which are allowed in the 

classroom, but is probably doing this to a greater -degree than necessary. 
= (Going to the bathroom, sharpening pencils, etc. This, is related to the 
.. teacher's monitoring or to poor planning of prJcedures, p rhaps.) 
viii. The students are wasting- time waiting for soJthing to be.done, because ' 
the teacher has not set up a routine or prp4dure which would save time. 

^5 
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VP -never the observer can relate some of these reasons to individual 
students or subgroups of students such as lower achievers, students whose view 
of the teacher is blocked, etc, it would be a very valuable thing to include 
in the narrative. It is possible that the procedure for doing the narrative 
may be revised later in the year to allow careful focusing on individual 
students or subgroups of students. The ideas provided in the narratives up to 
this point can be utilized then in asking particular questions about these 
individual students or subgroups. 
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GUIDELINES FOR FILLING OUT THE TIME LOG 
.After the observation is completed, the observer should fill out 
the time log on the forms provided* This would include completing the 
identification field at the top of each page, and describing each 
interval of time in terms of the Start Time, Stop Time, number of 
students involved, and a brief description of the activity. The time 
log has space for three concurrent activities to be noted, so that it 
will be possible to describe the different schedules of various groups 
in the classroom. If the class meets as a whole for the entire period 
then only the left hand set of columns will be filled. This time log 
should include intervals spent in transition and dead time, as well as 
intervals of time spert in actual instruction and work on assignments. 
If five or more students are in dead time or transition while the rest 
of the cl2>ss is working on something else, then they should be placed 
in a separate column on the time log. When the time leg is completed, 
it should be possible to follow the schedule throughou:: the class 
period for all of the students in the class. This does not h:«an that 
it should be done in terms of individual students, but rather in 
numbers of students in each subgroup who are following different 
schedules. An example of a completed time log is attached. 

If the observer has more than three activities to note because 
four or more groups are following different schedules, s/he should use 
an extra page, rather than trying to squeeze all four on to the three 
sections of one page. When completed, the pages ^hould be numbered 

consecutively and noted as "Page of If there is a 

continuation on a second page for a fourth group, that page should be 
numbered "2a", etc. 




Teacher # School # Observer # 



Stc^rt Stop # Subject/activity 



SAMPLE FORM: T I M E L 0 G 
Date ^ AM PM # in class Page of 



Start Stop # Subj ect/activity Start Stop # Subject/Activity . 
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GUIDELINES FOR NOTING 
STUDENT ENGAGEIIENT RATINGS 

At fifteen minute intervals, the observer should complete a Student Engagement 
Rating. This consists of 1) four kinds of information about the classroom context 
at that time, and 2) the number of students who can be classified in eacb of 
eight different categories of engagement. The observer should complete the first 
rating within the first 15 minutes, and then maintain a 15-minute interval between 
all subsequent ratings. 

Descrifation of Classroom Ccut:ext . ' 

In order to provide information about the context in which the engagement 
rating was taken, the observer should code the format of the classroom, the 
subject matter on which the teacher was focusing, the subject matter on which 
most of the students were focusing, and the number of students in the class at 
the time the rating was taken. ° " j. 

Classroom Format.. There are 18 categories available to describe the format 
of the classroom. The first 14 of these categories describe ways in which the 
teacher may organize the class for instruction. Basically, the 14 categories 
represent combinations of the following three variables: 

—The focus of the teacher's attention; that is, to which part of the 
class she is directing her attention and whether she is actively teaching some- 
thing, or simply monitoring the students' progress on independent work. 

—The focus of the students' attention at the time; that is, whether every- 
one in the group is paying attention to a single stimulus, or whether each 
student is attending to something individually at his or her desk. In the 
of individual assignments, it is important to note whether the students are 
working on the same assignment or whether they have different assignments. 
—The number of adults who are actively supervising students. 

III-19 20X 



case 



page 2 

Classroom formats 1 and 2 are similar in that they both have the teacher 
focusing his/her attention on the entire class at once by teaching something 
to them, and the students' attention is therefore supposed to be directed toward 
the teacher or something else of central importance. For the first two catego- 
ries, it is assumed that only one adult is involved. In the case that two adults 
are involved presenting. a single lesson to the entire class, the format would 
still be coded as belonging to one of these two categories. The difference 
between formats 1 and 2 is their content — whether academic or procedural. 

1. Teacher presentation to whole class, academic in nature . In order to 
be classified as this format, activities in the room at the time must meet the 
above description, and in addition must focus on academic content. Some 
examples of this are: the teacher lecturing to the whole class; teacher asking 
questions and responding to answers from the whole class; teacher giving a 
science demonstration; teacher reading aloud a story to all of the students; 
teacher working at the chalkboard; teacher leading a spelling or math drill; 
teacher using an audio-visual aid such as overhead projector, film, or television 
to which the students are expected to attend; presenting assignments to the 
entire class; and checking work with the entire class. If four or fewer students 
$ire doing something different while the rest of the class is being taught in 
this fortnat, then the category should still be used. 

2. Teacher presentation to whole class, procedural/behavioral . To be 
coded in this category, the activities in the classroom must fit the description 
given above, and the topic must be classroom routines, procedures, rules, or 
behavior. This format is used frequently in the first few weeks of school. 
Examples of it might be describing to the entire class the way that they are to 
wear their name tags during the week, how they are to care for their books, how 
they are to enter the room in the morning, sharpen their pencils, go to the 
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bathroom and get water, how they are to hand in their work each day', how they 

t 

are to make transitions in the room or line up to leave the room, and presentJ- 
tion of an attention-getting signal to the entire class with an explanation of; 
what it means. 

Format categories 3, 4, and 5 are similar. in that the teacher's attention 
is being focused on a sub-group (usually a small one) while other students in 
the class are working^ on independent assignments without any adult other than 
the teacher supervisihg them. Thp students who are not with the teacher are 
called "out-of-group" students. The differences between categoriss 3, 4, and 5 
have to do with whether or not these out-of-group students are working on same i 
or different tasks. , | 

?' Teacher presentation to a small group with others in class working ! 
independently wit hout adult supervision . All out-of-group students are" on same l 
task. This means that every out-of-group student is supposed to be working 
independently on an academic assignment and everyone is working on the same 
assignment. A typical example is for the teacher to be having a reading lesson 
with a small group while everyone else in the class is supposed to be completing 
the same ditto sheet. 

^- Teacher presentation t o small group with others in class working 
independently without adult super vTsTon. Out-of-group students are on more ^h.n 



one task but all of their assignm ents ere, within the same content area . A 
typical example of this is the teacher working with a reading group while other 
students in the class are also engaged in reading activities. Half of these 
out-of-group students are supposed to be -working with SRA materials, while the 
other half are supposed to be working on reading worksheets. 

5. Teacher luesentation to sm\71 group with others in class working 
Independently without adult supervi sion. Qut-of-group students are on mor. 
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than one task in more than one subject matter area . A typical example of this 

is the teacher working with a small group and listening to them read aloud while 

some of the out-of-group students are working on handwriting assignments, and 

while others are working on math assignments. 

Categories 6, 7, and 8 are similar in that the teacher's focus of attention 

is not directed toward teaching a lesson per se , but in circulating around the 

room to check on students who are working on academic assignments. The teacher 

is the only adult responsible for monitoring the students' work. The differences 

■ i *i 
between categories 6, 7, and 8 have to do with whether the students are grouped 

or not and whether they are working oasame or different assignments - 

6. Independent individual activities . Each student is focused on his or 
her own individual work while the teacher circulates through the room . All of 
the students are working on the same task . An example is the . teacher telling 
everyone to work the same set of ten problems out of the math book, and then 
circulating around the room while they are doing it. If the teacher is actually 
conducting a class discussion by interspersing brief questions with written 
answers which are immediately discussed, the format is not coded as a 6^, but 
would instead be coded as a 1-^ since the focus of the lesson would be the class 
discussion. The focus of format j6 is on the students all completing the same 
assignment, but completing it independently. 

7. Independent individual ao^tivities . Each student is focused on his or 
her own individual work, while the teacher circulates through the, room . The 
stu dents are on different tasks .' The^ only difference between this and Category 
6 is that more than one assignment has been given to the students, so that not 
all .of the students are doing the same assignment at the same time. There may 
be times when all of the students may be working on the same thing, and then 
some students gradually begin working on something else. The convention 
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here is that* when fewer than five students are doing something different from 
the rest of the class, the format should be coded as category 6^. ^i/hen five or 
more students are doing something different for their individual work, the 
format should be coded as category 7. 

^' Students are working in small groups with the teacher circulating 
through out the room going from group to group . At least five students are engaged 
in some kind of group activity such as playing a game or reading the parts of a 
play. They are doing this without the direct supervision of the teacher -except 
when he or she happens to monitor them. If assignments have been given by group, 
but the students in the group are not working with one another (i.e., they are 
still doing independent, individual work), it would be coded as a category 7^. 

Categories 9 through 14 have the same definitions as categories 3^ through 8^, 
respectively, except that another adult besides the teacher is in the classroom 
and is working with the students' on their assigned tasks. This other adult can 
be an aide, a student teacher, a student intern, or a parent volunteer. He or 
she must actually be working with students on academic tasks and not simply 
doing clerical work for the teacher at that time or observation. 

. Categories 15 and 16 are to be us^d whenever the student engagement .rating 
is taken during a transition. 

Transition, students staying in room and changing activities . This 
category' shoiiid be used when fewer than five students leave the room, and others 
within the room are moving between small groups or getting out new materials for 
a different subject or different activity. 

Transition, students entering or leaving the room . This category should 
be used when at least five students enter or leave the room for the purpose of 
changing classrooms for an academic activity. 
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17. .Dead time. This category should be used when the entire class (or all 
but four or fewer) has been left in "dead time/' The teacher has not given them 
any definite assignments or communicated any expectations to them about what 
they are supposed' to be doing. An example of this might be finishing, a public 
discussion five minutes before lunch and not telling the children what to do 
then, so that they sit at their seats. Another example might be students waiting 
for another class to come in to begin some lesson which is team taught. Typically, 
dead time for an entire class is a short period of time in which the students are 
waiting for some transition to begin. It "can also occur if the teacher is con- 
ducting a lesson and is interrupted or called aside, and she leaves the students 
without making provisions for their doing anything. 

18. Other . If there is some activity which cannot be described by the 
preceding categories, the observer should code the format as 18^. There., should 
be a clear description in the narrative of what was happening iri\ the class at 
that time. 

Subject-teacher focus . The observer should note the code number of. the 
subject matter on which the teacher is concentrating at the time. If the teacher 
is addressing the entire class or a small group, the subject matter is the one 
t^hat he or she is teaching. If the teacher ., is circulating about the room while 
the students are doing seatwork, the subject matter is that assigned to most of 
"the students in the room. A list of categories and code numbers^ir. given below. 

Subj ect-s tudent focus . The observer should note the code number of the 
subject matter on which the students are- focusing. 'In the event that che teacher 
is working with a., small group, this category applies to those students who are 
out of the group, working at their seats or at other centers. Many times the 
focus of the teacher and students will be the same. 

1D6 ■ 
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The subject matter codes are: 

1. Reading 

2. Spelling 

3- Handwriting 

4. Grammar or other aspects of Language Arts or "English" 

5. Reading/Language Arts (Assignments which av^ a combination of the two 

subject areas, including anything in categories 1^ through 4 above) 

6. Math 

7. Social Studies ^ ♦ 

8. Science 

9. Spanish \ 

10. Art 

11. Music 

12. Classroom procedures and rules 

13. Social-emotional, affective focus,. such as. Magic Circle 

14. Transition 

15. Dead time (See definition given for format //17) 

16. Other subject area ' ^ 

17. Mixed. A combination of two subject areas being taught at the same 

time other than the combination of reading and Language Arts, which 
is categorized as //5. An example would be math, spelling, and 
reading assignments being worked on at the same time. 

There may be occasions in which a single activity is occurring, but it isn't 
clear which subject matter should be noted, since two are actually involved in 
■ teaching some content. Examples of this are: learning to sing songs in Spanish, 
and a Science lesson that involves the application of math. In these cases, the 
observer should decide which is the primary focus of the lesson and categorize 
that. This is usually evident from the daily schedule or the teacher's announce- 
tnent about the activity. 

Vj7 
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Number in Class at Tlmo . This should be clie total numhor of students who 
were in the room and could therefore be considered in the Student EnEagement 
Rating, This ,may not represent the number of students attending class that day, 
since students may be in the bathroom or in other places in the school at the 
titne of the rating. The number noted here should be the total noted in eight 
categories of student engagement for that rating. 

Categories of Student Engagement 

Definitely Qn- Task , Academic . S tudents falling in this category are those 
working on an academic assignment or receiving an academic presentation, and who 
are very clearly paying attention to the task. That is, the observer is very 
confident that they are actually engaged in the academic activity which the 
teacher is expecting them to be engaged in and attending to. In order to be 
considered academic in nature, the students must be receiving new information 
from the teacher about some skill involved in reading, writing, spelling, grammar, 
math; etc., or some set of facts involved in these or other areas (e.g., social 
studies, science, music), or they must be using such skills or facts in completing 
an assignment. This category does not include instructions from the teacher 
about activities which are preparatory to beginning an academic task, or neces- 
sary for completing. an academic task, .such as those described under the two 
categories of "on-task, procedural," below. It does include activities after 
assignments which are related to academic skills — reading library books, coloring, 
etc. 

. 103 ^ . • . 
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Frohablv On-Task. Academic . Students failing in this category are those who 
are supposed to be working on an academic assignment or attending td'an academic 
presentation, but who cannot confidently be said to be attending; however, they 
are not definitely, off-task either. Students falling in this category might be 
those who are sitting at their seats with work in front of them but who are look- 
ing up at the wall or out the window at the time the rating is taken. The student 
might be thinking about the task, he might be resting momentarily before returning 
to work, or he might be daydreaming. The observer cannot tell by simply watching 
the student; however, it is also clear to the observer that the teacher would not 
be likely to correct the student for his behavior at that time; that is, it is 
not clearly off-task, unsanctioned behavior. 

Definitel y On-Task, Procedural . Students classified in this category are 
those who are clearly engaged in some procedural activity which is preparatory to 
beginning an academic activity, or is necessary for finishing it. Such activities 
include moving through transitions, sharpening pencils, getting out of new 
materials or putting up used materials, turning in work, putting headings on 
paper, collecting books from other students, finding one's place in a textbook, 
and listening to a teacher give an assignment, when this doesn't involve the 
teacher actually presenting new academic information. (For example, listening 
to the teacher explain that "Your math assignment is to do all of the problems 
on pages 72 and 73," would be on-task, procedural , but listening to the teacher 
say, "The way to add fractions is . . . ," would be academic .) Sometimes pro- 
cedural tasks involved the entire class (e.g., lining up to go next door for 
reading) and sometimes an individual will be doing something alone which can be 
considered procedural (such as turning in paper). It als^- -'ncludes class 
procedures such as lining up for lunch or dismissal, collecting money from 
students, or any other procedure initiated by the teacher for the sake of getting 
something done. . 
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Probably On-Task, Procedural . Students classified here are those whom you 
think are probably engaged in some procedural activity, but who are not clearly 



of this would be a student who is moving across the room, and you suspect that 
he is going to a skill box to pick up some materials, but it is not absolutely 
clear to you that he is doing this or just wandering around. The same category 
would apply to someone who is waiting near a supply area or waiting near the 
teacher's desk, and you suspect that the wait is part of continuing some academic 
activity, but you are not absolutely sure. 

Of f -Task, Sanctioned . Students are to be classified here when, at the time 
of the rating, they are involved in some activity that is not academic or pro- 
cedural in nature, but which is allowed in rhe classroom. Typically, this 
involves going to the bathroom, getting water, and going to and from the waste 
basket . 

Off-Task, Unsanctioned . Students ai'i classified in this category when they 
are very clearly misbehaving and doing something which the teacher does not approve 
of. It is not essential that the teacher correct the student for them to be 

classified here. The definition of unsanctioned behaviors depends on the rules 
each teacher has established for his or her class, and therefore, what is unsanc- 
tioned in one room may not be unsanctioned in another. Typically, however, behaviors 
which would be classified here would be: talking to one's neighbor when this is 
not allowed, cheating on a test, playing around in a disruptive manner instead of 
working, and being out of one's seat when this is not allowed. 

Dead Time . Students should be classified here when the observer realizes 
that there is nothing specific which students are supposed to be doing and when 
they are not engaging in unsanctioned behaviojr. This would include students who 
are waiting for a transition as part of the whole class and students who have 
finished all of their assigned work and who have not been given anything else to do. 



doing so. However, they are not obviously off-task or misbehaving. An example 
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No Datia (Can' t: See) . If there are studer.cs in the classroom who cannot be 
seen by the observer, they should be included in this category. This would include 
those who are working behind dividers and any student whose back is to the observer 
•when it is necessary to see the face in order to make an accurate rating. This 
category would not include students who were out of the room at the time the rating 
was taken, since these students are not counted in the "Number in Class at Time" 
for that particular rating. 
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Student Engagement Ratings 
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Teacher // 



School // 



Observer !r 



Date 



AM PM // students 



Page 



of 



Time 












Format 




i 








Sub j . -T. 












Subj . -St* 












// in room 












// def* on, acad. 












//prob. on, acad. 










/ 


// def, on, proc. 












// prob, on, proc. 












// off, sane. 








■ 




// off, unsanc. 












// dead' 










// can't see 












6 7 8 9 . 10 



Time 












Format 












Subj , -T* 












Subj. -St, 












// in room 












// def. on, acad. 












// prob. on, acad. 












// def. pri, proc. 












// prob. on, proc. 












// of f , sane. 












// off, unsanc. 












// dead 












// can't see 
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Reliabi I ity 

During the week. February 12-16, a series of reliability 
observations v;ere done using the four regular observers. Pairs of 
observers observed a class and did Student Engagement Ratings at the 
same time but without consultation. Component Ratings were done by the 
observers following each reliability observation. These pairs of 
ratings were used as a check on rater reliability. It was found that 
agreement between observers on both the Student Engagement Ratings and 
the Component Ratings was quite high. Reliability was computed on the 
32 variables on the Component Ratings using the intraclass correlation 
statistic. Only eight ratings had a reliability of less than .8. In 
addition, classroom narrative records were checked periodically by staff 
members for sufficient level of detail. 
Final Teacher Interview 

A final interview of the participating teachers took place in May, 
after the cessation of observations. The purpose of this interview was 
to gather information about relationships with other, school personnel, 
major changes made during the school year and desired changes for the 
next year, assessment of the students, and the organization of 
ar ithmet ic ins true t ion . 

In addition, teachers were given some information about the study 
as a whole and some individual feedback on the teacher's classroom 
organization. A copy of the questions and materials used in this 
interview follow. 
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Classroom Organization Study 
Exit Teacher Interview 
(May, 1978) 



Introduction should give an overview o£ the major types of questions to 
be asked, and should say that the interviewer also will provide 
information about the study as a whole and feedback to the individual 
teacher. 



Have the teachers sign the vouchers 1 1 Remind them that the checks may 
take several weeks, since they have to go through UT paperwork. Make>sure 
that summer addresses are correct. 



Working relationships with other adults in the^school 

1. IVhat contacts do you have with other adults in the school about students 
that you share? (Probe for information about music, PE, Resource, Title I 
librarian, etc., especially regarding joint, planning and any systems for 
reporting to one anothe.rO 

2. How often do. you discuss instruction and classroom organization with 
the principal?. 

Changes over the year, and desired ^changes, for next year 

3. We're interested .in the .major changes you may have made over the 
year in your room arrangement, e^ecially in student seating patt .s. 
What have yoii changed, and what were your reasons? Are you able to 
have -the arrangement you think is most ideal? 

4. We are also interestec! in major scheduling chatiges that have taken place. 
What have you changed, and what were your reasons? 

5. Have you made any major changes this year in terms of curriculum, 
classroom organization, etc. (when compared to last year or maybe when 
compared to expectations, for the new teachers)? How has this affected 
your teaching? ^ ^ . 

1m X 
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6. Are you planning any changes in curriculum and/or organization for 
next year? 



Students 

7. Are the pupils you are teaching this year markedly different from those 
you have taught in the past? In what ways? 

8. Are there students in your class who have missed a great deal of school? 
Do you have any special routines or procedures for their making up work? 
(Probe for >which subject areas are to be made up.) 

9. IVhat do your students know about the grades that are assigned on 
the report cards? Are they aware of what they will receive? Does 
this seem important to them? 

10. We are interested in knowing what materials were covered in reading 
this year, and how far the students got in the basal series. (Refer 
to accompanying form:, with student names, and note the beginning and 
end points for each student. Be sure to add students who have entered 
since the beginning of the year, and note those who have moved.) 

Arithmetic Instruction 

11; ; l\fhat is your daily procedure for teaching arithmetic? (Probe for schedule, 

assignments, what materials used, grouping practices and/or individualization, 
„ . feedback to students about their work) 

12.. Do you have students who are way below grade level or way above? How 
do you deal with them? 

13. Could you indicate how many of your students (roughly) have learned 
these concepts? (Refer to the accompanying form for math concepts.) 

Special students 

14. Are there particular students who have been memorable behavior problems 
this year?^ How did you deal with them, and how do you think that worked? 

15. Do you have students who leave your classroom for most of their 
instruction in reading and math? (Note on list with names.) UTio leaves 
for what other purposes? 



Teacher // 



SAMPLE FORM: Beginning /fcnding Keaaer List 
School // Date: 





Beginning 

p p el PT 
rv tL. &i Li i. 


Ending 
Rend ex 


Reading 
Tea cliGr 


Math 

T'rj 1 r» V» n y* 
i licl L. n v: L 
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ECONOMY 



Level 3 
Pug PIM 
Sun Tree PP. 2 
Zip! Pop! Go: 
PP.3 



HAR COURT 
CRACI: 
JAVANOVICH 



Level 3 

Sun Up 

A Happy Morning 
A Magic Afternoon 



iIOUGiiTON 
MIPFLIN 



HOLT, - RlhjEHART 
AND WINSTON 



Level 3 
Tigers PP.l 
Lions PP>2 ' ^ 
Dinosaurs PP.3 



MACMILLAN 



Level 4 



Reinforcement 



Level 5 
Green Feet 



Level 5 
Sun 5 Shadows 



Level 6 
Blue Dilly Dilly 



Level 6 
Together We Go 



Level 7 
Curbstone 
Dragons 



Level 7 
A V/orld of 
Surprises 



Level 8 
Mustard Seed 
Magi^c 



Level 8 
Going Places, 
Seeing People 



Level 9 



Re teach 



Level 10 
Air Pudding f, 
Wind Sauce 



^ Level 10 
Widening Circles 



Level 11 
Mysterious 
Wysteria 



Level 11 
Ring Around 
the World 



Lev^l 12 



Ret each 



Reinforcement 



Leyel 5 
Rainbows 



Level 6 
Signposts 



Level 7 
Secrets 



Level 8 
Rewards 



Enrichment 



Level 10 
Panorama 



Level 11 



Fiesta 



Enrichment 



Level 13 
Kaleidoscope 



Level 14 



Images 



Level 15 



Level 13 
Time to Wonder 



Level 13 
The Magic V/ord 



Reteach 



Enrichment 



Level 15 v 
Freedom' s Ground 



Level IS 
Bold Journeys 



SA>IPLE-raRM 

Teacher // School // Date: 



MATHEMATICS COMPETENCIES 





Didn't 
Teach 


Few 
1-3 


Some 


One- 
Half 


Most 


1 

All 


1. Recognizes and names halves, thirds, 
fourths, fifths, sixths and eighths 
of a region or set. 














2. Computes cost of items in dollars 
and cents. 














3. Tells time to minutes; knows 60 
seconds = 1 minute. 














4. Reads Fahrenheit and Celsius scales 
on a thermometer. 














5. Reads and writes any three-digit 
number. 














6. Counts by twos, fives and tens to 

100, and by threes to 30 and fours 
to 40. 














7. Orders and writes numerals to 1000. 














8. Uses place value, terms of "ones," 

. "tens," "hundreds," and "thousands" 
to describe a number. 














9. Knows family of facts for addition 
and subtraction 0-18. 














10. Adds and subtracts three-digit 
numbers with regrouping. 














11. Knows the multiplication tables of 

0, 1, z, 3, 4, 5, 10 and understands 
the process. 














12. Divides a two-digit number by 1 - 5. 














13. Uses. the terms "greater than," 

"less than," "equal to," "plus,", 
"minus," "quotient," aiad "product" 
with the appropriate symbols. 










- 




14. Uses knowledge of numbers and selects 
the correct operation to solve 
written and oral problems which 
■involve concepts related to geometry, 
measurement and numbers. 














15. Completes charts and tables to show 
understanding of a rule or pattern 
in addition, , subtraction and 
multiplication. 















What page did you get to in the book? 
Did you' skip around? 
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Preliminary Results 
The Classroom Organization Study 
The Research and Development Center for Teacher Education 
The University of Texas 

After our initial review of data collected in the first three weeks of school, we 
listed five characteristics of teachers who had greater »'time on-task" and less 
'*tirae off-task." By "time on-task," we meant that the students appeared to be doing 
what was expected of them, such as working at assignments or going through classroom, 
routines. By "time off- task," we meant that the students were occupied in activities 
that took them away, from their work, either misbehaving or moving around the room for 
personal reasons (e.g., getting water). 

The average class had two kids off-task every time we looked (every 15 minutes). The 
range across classes was from .5 students off-task each time we looked to about 6 stu- 
dents off-task. 

We averaged each class's ratings for the second and third weeks of school, and then 
looked at what the teacher had done during the three weeks to set up the classroom.' 

The following points seemed to us to differentiate the teachers with more time on-^task 
(and less of f-task) f rom those who had more problems with students being off-task more 
of the time. We will examine the data collected over the rest of the year to see if 
these points hold up as general recommendations. Each point can be illustrated with ^ 
specific examples, both positive and negative. Materials that will be produced from 
the study will include these specific examples (anonymous, of course), and you will 
receive these when they are ready. 

1. They demonstrated an ability to analyze the tasks of the first few. weeks of school 
in precise detail. Their presentations to the students about rules, procedures, and 
assignments were very clear, and they followed up on these by pointing out to the. stu- 
dents in detail what they were and were not doing that wa:s appropriate. They were very 
consistent in this attention to detail. ' 

2. They had thought in advance of the rules and procedures necessary in their classrooms, 
and saw that these were established and communicated to the students before the problems 



arose. 



3. They considered that the teaching of 'these rules and procedures was a very impor- 
tant part of instruction those first few weeks. That is, they taught ''going to school" 
skills by providing^practice in moving through" procedures , responding to, signals , . etc. , 
and pointed out . to the students when they were behaving appxopriatelly . They spent as 
much time as was necessary to establish basic routines and smoothlyj' running procedures. 

4. They were. able to "see through their students^ eyes" in planning the classroom and 
introducing the students to the school year. That is, they could predict what would 
confuse or distract a student, and what would be of immediate concern to him or her. 
They used this information in such a way that problems were prevented. 

5. They introduced their students to independent work gradually, and did not "let go" 
of them without close supervision until they had communicated certain expectations to 
the students and had established credibility as the leader of the classroom. 
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Observer Outline for the Classroom Organization Study^ 
Research and Development Center for Teacher Education 

The following outline was used by the classroom observers to gather 
information in each of the third grade classrooms taking part in the study 
this year. The outline is divided into two very important parts, those 
activities involving teacher organization of the classroom and those involv- 
ing student behavior in the classroom. 

I. Teacher Activities (Those activities engaged in by the teacher to establish 
and maintain the classroom organization,) 

A. Preparation (What the teacher has done to get ready for that day- 
planning and preparation.) 

1. organization of materials and supplies . 

2. information presented to the students ■ 

B. Teacher communication to students (How and what the teacher communicates 
to the students about what they are supposed to be doing.) 

1- directions, explanations, and corrections delivered to the students 
about their activities and use of materials 
, , 2. teacher's use of praise and . criticism (particularly the antecedents 
and consequences of the use of praise and • criticism) 

3. academic activities and use of materials in them 

4. procedural activities and use of materials in these 

5. behavioral corrections about unsanctioned activities or unsanctioned 
use of materials , 

C. Teacher contacts with student^ and monitorint^ of classroom (How the 
teacher knows what is going on-with each student.) 

^.1. academic contacts (how the teacher selects students to answer in a 
public setting, such as a question-and-answer discussion; how the 
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teacher maintains contact with students who are doing private work, 
seatwork, or working alone; how the teacher gives feedback to the 
students about mistakes made on their written work) 

2. procedural contacts (monitoring the students they go through 
pre-established routines, and procedures) 

3. behavioral contacts (how quickly the teacher responds to misbehavior 
by the students; whether the teacher's contacts with the students 
are primarily academic, procedural, or behavioral contacts in 
nature) . ^ ; 

^- Constrai nts on classroom organization (What gets in the way, what 
interrupted the routine, and how the teacher deals with this.) 

1. interruptions of classroom routine (i.e., internal or external) • 

2. general conditions in the classroom or school that might make the 
teacher's job more difficult ' 

3. having insufficient room for children to v hang their coats 

4. whether. or not the teacher has the power or authority to do certain 
things, or has access to resources that would solve the problem 

II. Student Activities and Response to Teacher 

A. Counts of students engaged- in various categories 

^' Students who are on- or off -task and the different reasons why 



ERLC 



1 •"•O 

J. . 

. III-3§. 



Other Data Collected From Observers at the End of School 

Each observer saw from ten to sixteen teachers approximately four 
or five times each, and each teacher was seen by at least two observers. 
Following the cessation of observations, observers did Summary Component 
Ratings on each of the teachers they had observed. The rating system 
was changed slightly, such that a score of seven meant that item was 
very characteristic of that teacher, a score of four meant that item was 
somewhat characteristic of that teacher and a score of one meant that 
item was not at all characteristic of that teacher. In addition, 
observers rated their teachers on other summary rating scales. These 
scales measured variables covering a wide array of classroom behaviors. 
The purpose of obtaining these final ratings was to obtain as much 
supplementary information as possible, and to provide leads for future 
research efforts. A copy of these ratings follow. 
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Teacher v 



Observer v 



' Date 

1. Assuming that the teaclier made the most efficient use of the classroom's space, 
** how crowded was this room? 

1 = Veiy crowded, inadequate space for number of students 

2 = Sufficient desks, but no room for anything other than essential furniture 

3 = Adequate space for desks, activity centers, reading groups 

4 = Adequate space for desks, activity centers, reading. groups, rug area 
* • 5 = Spacious room, no area seemed cramped 

2. Simply comparing the number of students and the dimensions of the individual 
** room, how crowded was this room? 

1 = Not at all 

5 = Very - j • . 

3. Uliat was the general comfort level of the room throughout the year? 

** 1 = Poor. Air-conditioning or heat frequently inoperable, situation in high 
noise area. Combination of circumstances made it uncomfortable 

2 = Slightly better. Only one environmental variable (ie. room temperature) 

was distracting 

3 = Average. No drastic extremes on either end 

4 = Good. Not even a consideration because things seemed fairly smooth 

5 = Excellent. Controls were very adaptable; near perfect -conditions 

4. How well did teacher utilize the space of the classroom? 

** 1 = Poorly; much space not used at all; heavy concentrations in particular 
areas " 

2 = Fairly; less than half :of room used with any consistency 

3 = Good; more than half of room used consistently 

4 = Better; most of room used at least once a week 

5 = Excellent; all parts of room used almost daily 

5. How adequate was space provided for student's belongings? 
** 1 = Not at all, closets were small, not enough room at desks 

5 = Very ' .. 

6. How adequate was space provided for teacher's materials and belongings? 
1 = Not at all 

5 = Very ^' 

7. To what extent was students' work visible in classroom? 
** 1 = None 

5 = Great extent 



8. How orderly was this classroom? 
** 1 = Not at all 

5 = Very 

9. How clean was this classroom? 
** 1 = Not at all 

5 = Very 



10. How attractive was this classroom? 
** 1 = Not at all 
5 = Very 

1 * • ^ 
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** - denotes that the item was reliable p = .05 
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11- How related (purposeful] to students* tasks and interests were bulletin boards? 
1 = }lot at all 
5 = Very 

12. How often did bulletin boards change? 
1 = Rarely 
5 = Frequently 

15. HoK well did teacher position the small groups [eg readiTig) to prevent blind 
** spots from occurring, regardless of deliberate private areas? 
1 = Poorly; she couldn't see everything 
5 = Excellently; she could see everything 



IVhat was the usual procedure for students leaving their seats while the 
teacher was involved with a small group? (getting materials) 

14. ( ) They could go one by one, by sex, without permission 

15, ( ) They xould go one by one, by sex, with permission 
16- C ) (omitted) 

1*7. ( ) (omitted) 

18- ( ) A few students could leave their seats without permission 

19- ( ) No established number could leave their seats without permission 

20. ( ) Other; specify: 

21. How often were the rules regarding movement enforced? 



1 = Never 
5 = Consistently 

22. How often were the rules regarding movement broken? 
** 1 = Never 

5 = Consistently 

Was there a procedure established for students entering the room in the morning? 
25. ( ) No, there was no procedure 

24. ( ) If there was one, it was not evident to the observer 

25. ( ) There was a procedure, but it was infrequently used 

26. ( ) There was a procedure; it was used half of the time 

27. ( ) There was a procedure.; it was used all the time 

Was there a procedure established for lining students up to leave the room? 

28. ( ) No, there was no procedure 

29. ( ) Yes, there was a procedure, but it changed all the time 

30. { ) There was a procedure, but it was infrequently used 

31. ( ) There was a procedure; it was used half of the time 

32. ( ) There was a procedure; it was used all the time 

35. Uliat was the, degree of orderliness by students in lining up to leave? 
** 1 = Low, they were never quiet or still 
5 = High, they were always quiet and still 

34. Could students predict the order of lining up? 
1 = Never 

5 = Always or almost always 

How often did the teacher use a punishment (staying after school, losing 
recess) when students were misbehaving in line? 

55. ( ) Frequently, all year * round 

56. { ) Frequently at the beginning of schooL ^.^^ 
37. ( ) Moderately, all year 'round 1.^0 

^ 38. ( ) Moderately, especially at the beginning of school 
ERsLC ^ ^ Infrequently^ 
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40. How often was the order of lining up used as a reward? (i.e., teacher chooses 
1 = Never quietest table to line up first.;) 

5 - Frequently 

41. How often did teacher call a student's name to correct an improper behavior 
. in lining up? 

1 = Never 

5 = Frequently 

42. How often did teacher send a student to his seat for improper behavior in line-? 
1 = Never 

5 = Frequently 

43. How often did the teacher send the whole class to their seats for improper 
behavior in line? 

1 = Never 

5 = Frequently 

44. How oftf^n was a systematic procedure used to select students to be classroom 
monitc .J or helpers? 

1 = Never 

5 = Frequently 

45. How predictable to the students was this selective process*? 
1 = Not at all 

5 = Very 

46. How many students were involved in being monitors? 
1 = Only a select few 

5 = IVhole class 

47. How well were routineSestablished so that room ran with a minimum amount of 
** interruptions; room seemed to run automatically? 

1 = Not well at all 
5 = Extremely well 

48. How often was teacher late in going places? 
1 = Never 

^5 = Frequently 

49. How often would teacher point out the time to students? 
1 = Never ' 

5 = Frequently 

50,. How often would teacher use the clock to obviously pace students' lessons? 
1 = Never - 
5 = Frequently 

51.. How often would teacher use a timer to pace students' lessons? 
** 1 = Never ■. 
S = Frequently 

52. Was the daily schedule displayed during the first three weeks of school? 
ND 1 = No 2 = Yes ^ ' 
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55. How ofen.was a schedule of daily assignments (e.g. by groups) posted? 
1 = Never 
5 = Frequently 

54. Was the daily schedule displayed at any time during the rest of the year? 
1 = iNever 

5 = Frequently 

55. How consistent was the morning schedule in terms of activities? 
** 1 = Not at all 

5 = Very ^ 

56. How often was the first activity of the morning affectively oriented 
(sharing, singing, magic circle...)? 

1 = Never 

5 = Frequently 

57. How often was the first activity of the morning procedural in nature 
** (calling the roll, flag salutation) ; 

1 = Never 

5 = Frequently 

58. How often was the first activity of the morning academic in nature 
** (the students entered the room and had an assignment to 

begin)? 
1 = Never 
5 = Frequently 

59. How often did you observe activities that were strictly affective in 
** emphasis (sharing, magic circle)? 

1 = Never 

5 = Frequently 

60. \\liat was the usual length of time between the time the bell rang and when 
** the teacher first addressed the class as a whole? 

1 = Ten minutes or more 

2 = Between five and ten minutes 

3 = Between three and five minutes 

4 = Between one and three minutes 

5 = Less than one minute 

61. How self-directed were students in entering the room, getting their things 
** put away, and getting ready for school? 

1 = Not very 
5 = Very 

Uliat was the teacher's usual first morning academic activity? (content) 

62. ( ) Spelling 

63. ( ) Handwriting 
64.. ( ) Math 

65. ( ) Reading 

66. ( ) Other 

Miat was the teacher's usual first morning academic activity? (format) 

67. ( ) Uliole group activity 

68. ( ) Small group with teacher, large group on its o\m 

69. ( ) Individual seat work, teacher circulated 
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70. How long did this first academic, period usually last? 
** 1 = Less than 15 minutes 

2 = Between 15 and 50 

3 = Between 30 and 45 

4 = Between 45 and 1 hr 

5 = More than 1 hour 

71. At what point in the morning did the. students first leave the room 
** (for PE, music, normal bathroom break)? 

1 = 8:50 or before 

2 = S:50 - 9:30 

3 = 9:50 - 10:30 

4 = 10:30 - 11:30 

5 = 11:50 - lunch 

# 

Iftiatwasthe bathroom procedure during the morning activities? 

72. C ) Class goes to the bathroom as a whole 

73. C ) Students go to bathroom within room according to set procedure 

74. C ) Students go to bathroom outside room according to set procedure 

During the morning actiyities, what was the established system for 
bathrooming? ^^^^^^^ 

75. C ) None, class^ goes as a whole 

76. ( ) Students must raise hands and get permission each time 

77. ( ) Students use token system (tags, cards) 

78. ( ) Students simply watch to see if bathroom is occupied 

79. ( ) Sign lip sheet 



80, 



Did this teacher nave reading groups? 

1 = No 

2 = Yes 



81. Kow many did she have? 

Specify number: 1234567 

ifliat was the Physical structure of the reading groups? (check all that apply.) 
^l. C J Set space, children bring their o\m chairs i^i / ^ 

83. C ) Set space, chairs permanently there; this area also used for 

other things at different times of the day 

84. C ;) Used students' desks 

85. C ) Set space, kids on floor or rug 

During the reading group (if there was one), what was the usual format 
ot the work that occupied the large group? (Check all that apply.) 
i56. C 3 Everyone doing the same thing at the same time 

87. ( ) Assignments according to reading group 

88. ( ) Individual contracts 

89. ( ) Mixture of two of the above 



90. 



At what point in time did this class as a wlvole tend to lose its 
effectiveness or concentration (compare to coming' down from a plateau)? 
1 = 9:00 or before (actually, never really got it together) 



2 = 9:00 to 9:30 

3 = 9:30 to 10:00 

4 =• 10:00 to 10:30 ' 

5 = 10:30 to 11:00 

91., Did teacher select reading groups in the same order every day? 
1 = Mo 



2 = Yes 



ICS 
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92. How much time elapsed between the time the teacher called for a reading 
group and its beginning [the teacher being able to address this group 
as a whole)? (Estimate your answer.) 

1 = Fiv^e minutes or more 

2 = Three to five minutes 

3 = Two to three minutes 

4 = One to two minut.es, _ 

5 = Under one minute 

Wliat was the size of the smallest reading group? 

93. ( ) Ten or more 

94. C ) Seven, eight, or nine 

95. ( ) Five or six 

96. ( ) Three or four 

97. ( ) One or two 

IVliat was the size of the largest reading group? 

98. ( ) Fifteen or more 

99. ( ) Ten to fifteen 

100. ( ) Seven, eight, or nine 

101 . ( ) Five or six 
102- ( ) Four or less 

105. How many materials did students have to bring to the reading group 
(books, paper, workbooks, pencils, spirals)? 
1 = Nothijag-^ 
^J^^Alot * 

Did students have to make many stops to pick up materials on the way 
to the reading group (this is trying to determine the amount of traffic 
coming and going to reading groups)? 

104. ( ) No 

105. ( ) Yes, one 

106. ( ) Yes, two or more 

107. Did this teacher exchange students with another teacher during the 
morning pejriod (teaming)? 

1 = No 

2 = Yes 

108 ^^*ere exchanged students mixed in with the other students in the seating 
arrangement? 

1 = No ' 

2 = Yes . ' • 

109. How' consistent was the afternoon schedule? 
1 = Not very; every day was different 

3 =. Fairly; things were consistent for at least weeks at a time 
5 = Extremely; never changed 

110. Uliat was the noise level of the classroom in general on a day to day bas 
★ ★ 1 = High. Alot of talking, moving around 

5 = Low. Very little if any ^ - 

Select the most generally true statement: ■ ■ ^ 

111. ( ) There was alot of talking and moving around in both morning and 

afternoon sessions 

112. ( } Tliere was more talking and moving around in the morning than in the 

afternoon 

12D 
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113. ( D There was more ■ talking and moving around in the afternoon than in 

the morning 

114. ( ) There was little talking at any time 



1 



115. How much social chitchat went on between students when they were supposed 
** to be working? . ^ ' 

1 = Could almost always see at least 5 pairs talking 
3 = Could occasionally see 1 pair talking 
5 = Rarely saw pairs talking 

116. How frequent was tattling by one student on another to the teacher? 
1 = Often, once a day or more. 

3 = Occasionally 
5 = Rarely 

117. IVhat was the teacher's usual response to tattling? 
1 = Acknowledge every time 

3 = Sometimes acknowledged 
5 = Never acknowledged 

118. How often did teacher use ignoring as a response to unsanctioned behavior? 
1 = Never 

5 = Frequently 

119- How often did teacher use citing the rule as a response...? 

** 1 = Never 

5 = Frequently 

120. How often did teacher call out student's name as a response...? 
1 = Never 

5 = Frequently «» 

121. How often did teacher order a student to stop unsanctioned behavior? 
■** 1 = Never 

5 = Frequently 

122. How often did teacher express feelings about misbehavior? 
** 1 = Never 

5 = Frequently 

123. How often did teacher question student to gather information about 
** misbehavior? 

1 = Never 

5 = Frequently 

124. How often did teacher isolate or s^eparate student for unsanctioned behavior? 
** 1 = Never 

5 = Frequently 

■* - ^ 

125. How often did teacher punish in some way students for misbehavior? 
1 = Never 

5 = Frequently 

126. How often did teacher non-verbally (eyeball, finger-snap) respond to 



misbehavior? 
1 = Never 
5 = Frequently 



130 

. TT.I-A7 
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127. In the observer's opinion, how often did the teacher let the class 
** get out of hand, or beyond control? 

1 = Never 

5 = Frequently 

128. Miat was the time period of the most frequent infractions? 
** I ^ 8:00 - 9:50 

2 = 9:50 - 11:00 

3 = 11:00 - Lunch 

4 = Lunch - 1:00 

5 = 1:00 - 2:30 

129. What was the average tone of voice used in correction? 
** 1 = Low 

3 = Moderate 
5 = Loud 

130. How frequently was teacher loud in correcting students? 
** 1.= Never 

5 = Frequently 

131. How frequently was teacher soft in correcting students? 
1 = Never, 

5 = Frequently 

132. How accurate was teacher in targeting (identifying the right one) 
** students for disruptive behavior? 

1 = Never very; often missed and called down wrong student 
3 = Average was good; could spot general areas, but 

avoided calling students unless s/he was absolutely sure 
5 = Very accurate; always knew; sixth sense... 

How were rules established in the classrpom? 

133. ( ) Teacher dictated 

134. ( ) Class and teacher discussed arid agree together 

135. ( ) Class decided completely on o\m; teacher acted only as moderator 

of discussion 

136. How often did teacher use staying after school as a punishment? 
1 = Never 

5 = Frequently 

137. How often did teacher use extra assignments (writing] as a punishment? 
1 = Never 

5 = Frequently 



138. How often did teacher use scolding as a punishment? 
• ** 1 = Never 

5 = Frequently 

139. How often did teacher send a note or call parents as a punishment? 
1 = Never 

5 = Frequently 

140. How often did teacher send student to office (principal] as a punislimcnt? 
1 = Never 

5 V Frequently 1 ''^ 1 ' . 

Ji ^ JL 

Er|c 
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141.. How often did teacher i^se a discussion in private (no scolding] 
as a punishment? \ 
1 = Never ' \ 

5 = Frequently 

142- Mow often did teacher use staying in from recess as a punishment? 
1 = Never 
5 = Frequently 

143. How often did teacher use the removal or gaining of points as a 
** a punishment? 

1 = Never 

S = Frequently ' 

144. How often did the teacher use the loss of a privilege for the whole 
class as a punishment? 

1 = Never - 
S = Frequently 

145. How often did teacher use individual loss of privileges as a punishment? 
1 = Never 

5 = Frequently 

UTiat rewards were used by the teacher? (Checl; all that apply.) 

146. ( ) Classmates clap or cheer * 

147. ( ) Special privileges 
1.48- ( ) Praise 

149. ( ) Papers posted 

150. ( ) Waiver or reduction of assignments 

151. ( ) Symbols (Stars, smiling faces) 

152. C ) Tokens or points redeemable for other rewards 

153. ( ) Concrete (Candy, money, prizes) 

154. ( ) Jobs (Monitor, helper, leader) 

155. ( ) Public recognition 

156. ( ) Other, specify: 

UTiat provisions were made for what to do when assigned work was completed? 

157. (0 Books ' (Check all that apply.) ^ 

158. ( )■ Games, instructional 

159. ( ) Games, non-instructional 

160. ( ) Lisl^ening centers 

161. ( ) Filmstrips, picture files 

162. ( ) Coloring pictures 

165. ( ) Other art activities: painting, clay 

164. ( ) Science experiments 

165. ( ) Aquarium, pets 

166. (r) Other, specify: 

167. How many students used free time materials during an average observation 
1 = None 

3 = 10-12 students 
5 = All or almost all 
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What continscncies were placed on the use of free time materials? 

168. ( ) Students were, to have finished required amount of work 

169. ( ) Students were to have achieved certain grade, or meet prodesi gnatod 

standard 

170. f ) Students were to have been good, quiet during period of time 

171. ( ) None, the materials were used for group activities 

172. [ ) Other, specify: 

173.1\Tiat was the extent to which teacher accepted '^come-ups" (students 
approaching) while conducting a small group? 
1 = Frequently 
5 = Never 

174. How often were *'come-ups^^ observed while teacher was engaged with another 
** student or lesson, e.g. while teacher was circulating? 

1 = Frequently 
5 = Never 

175. How frequently did teacher respond to misbehavior in the rest of the 
class when conducting a small group with getting up and going to 
student? 

1 = Often • ' 

5 = Never 

176. How frequently did teacher respond with non-verbal cues (eye-balling or 
** finger-snaps or pointing) to jiisbehavior in the rest of the class when 

conducting a small group? 
1 = Often I 
5 = Never 

177. How frequently did teacher respond with verbal acknowledgement (calling 
name) to misbehavior in the rest of the class when conducting a small 
group? 

1 = Often 
5 = Never 

17S.H0W frequently would teacher call students to com e up in response to 

1 = Often 
5 = Never 

179. How frequently would teacher use a signal (bell) in response to 

**' 1 = Often 

5 = Never 

When the teacher left the small group to deal with something in the room, 
what usually occurred in the small group? 

180. ( ) Off-task, noisy 

181. ( ) Off-task, quiet 

182. ( ] Remained on-task 

1S5.M0W often did the teacher deliberately teach "following instructions?" . 
1 = Never 
5 ' = Frequently 

i '1 O 
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184. How often ^id teacher present instructions in more than one way (speaking 
** as well as placing on board)? 

1 = Never 
S = Always 

185. In giving instructions, how often did teacher question to deterrriine the 
** extent of students' understanding? 

1 = Never 
5 = Always 

186. Did the teacher usually give adequate, easily understood instructions? 
** 1 = Never 

S = Always 

187. Did. the teacher give overly explicit, repetitive, poorly paced instructions? 
** 1 = Never i ^ i 

5 = Always 

188. Did the teacher display patience in elaboriting instructions? 
** 1 = Never 

5 = Always 

189. How often did teacher get students* attention before speaking; and not 
** try to talk over them? 

1 = Never 
5 = Always 

150. Did the teacher have an established' signal for students to begin their work? 

1 = No 

2 = Yes . 

191. Did the teacher have an established signal for getting students' attention? 

1 = No 

2 = Yes ■ . 

1Q9 f f ^^^^^ ^^^d to get students' attention? (Check all that apply J 
• I J Bel 1 . 

193. ( ) Lights, flicking 

194. ( ) Going to a certain spot in the room 

195. ( ) Verbal C*'boys and girls'^ counting) 

196. ( ) Physical (holding fingers to lips) 

197. ( ) Other, specify: , . 

198. How -consistent was teacher in using her signal? 
** I = Not very 

5 = Very • 

199. How consistent was the success of the signal in getting students' attention? 
** 1 = Not very 

5 = Very 

200. How often in a typical observation did students have a choice in choosing 
their assignments? 

1 = Never 

5 = Frequently 

7 o « 
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Where was the usual location for individual help between student and 
teacher? 

201, i ) Teacher's desk. 

202, ( ) Student's desk 

203, ( ) Work table 

204, ( ) Reading circle 

205, ( ] Other, specify: 

206, Did the teacher usually bend down to student's level during talk? 

1 = No 

2 = Yes • 

207, Did you ever see students work in cooperative groups (either for academic 
or art or other kinds of activities^? 

1, = No 
2^= Yes 

If yes, estimate how many times: 



208. How often did you see peer tutoring situations in a typical observation 
(any time when students were allowed to help each other^? 

1 = Never 

5 = Frequently 

209. How well did teacher accept students' ideas and/or integrate them 
** into class discussion? 

1 = Teacher rejected all or almost all student input; resented it 
5 = Teacher was receptive during discussions and encouraged it 

210. To what extent were students expected to care for their own needs 
(academic, instructional, as well as personal^ without getting permission? 
1 = To no extent 

5 = They were completely on their own; under no circumstances short of death 
were they to ask the teacher what they could figure out for themselves. 

211. What was the level of student talk during individual activities? 
** 1 Low^ 

5 = High 

212. How often did students approach teacher, leaving their desks, when they 
** needed help? 

1 = Never 

5 = Frequently ** 

213. How often did students raise their hands when they needed help from 
the tdacher? 

1 = NeVer 

5 = Frequently 

214. How often did students call out when they needed teacher's help? 
1 = Never 

5 = Frequently 
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215. wiion students luul trouble with tlialr work, how oftan was it ohsorvoci 
that thoy copiod fvoin no i .uhbors ? 

1 = Novor . 

5 = Frequently 

o 

216. When students had ti\oublc with their work, how often was ;it observed 
that thoy sought help from teacher , either wait ins at desk, or coiiiinp 
up • 

1 = Never 

S = Frequently ' 

217. When students had trouble^with their work, how often was it observed 
that t-hey sought help from peers or neighbors (not just copying,;)? 

1 = Never 

5 = Frequently " 

218. i^fiien students had trouble, how often did they just sit and do nothing 
** (Do not count sitting, waiting for teacher to get to theml? 

1 = Never 

5 = Frequently 

219. To what extent, did students act up when teacher left the room? 
** 1 = a bunch; a regular circus 

5= Not at all] very self-disciplined 

220. How. much/of the class was usually part of the "action'' when teacher left 
the room? 

1 = All! 

2 = Three-quarters 

3 = Half . ' ^ 

4 = One-quarter 

5 = A few 

221. What was the teacher's usual response to call-outs during a whole class 
** discussion? 

1 = Always respond 
3 = Sometimes respond 
5 = Never respond 



222. What was the teacher's usual response to come-ups during a whole class 
** discussion? 

1 = Always respond 
3 = Sometimes respond 
5 = Never respond 

r. What would most students do if left in dead-time for five minutes 
. with teacher still in the room? (Check all that apply.) 
( ) Act-out and' misbehave; walk around, talk loudly 

224. (; ) Tai]^ quietly with neighbor 

225. (; ) pind something to do at their desks 

226. How much self-control in remaining on-task- did most students exhibit 
** during a typical observation? 

1 = Not much at all 
5 = A lot 

13G 
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227. }|ow often did wandering occur that was obviously not task related? ' 

** 1 = A lot 

5 = i^ot much at all 

228. no;^ ^-jij teacher make use of students in performing some tcachi r 
fufictions? (Check all that api)ly.) 

228. ( ') Taking roll 

229. ( ) Reading spelling words to class 
( ) Grading papers or tests 

2M. ( ) Other, specify: 

232. ( ) Some combination of the above 

233. How democratic was the teacher's leadership style? 

** 1 = Teacher made all decisions and announced them to class as final. 
5 = Teacher frequently consulted class and allowed them to share in 
the decision making and planning 

234. To what extent did the teacher encourage a competitive atmosphere? 
1 = Not at all 

5 = A lot 

235. How much emphasis did the teacher seem to place on grades? (Either re- 
cognizing good grades, posting good work, especially if no place for 
other work . ) 

1 = None 

^ = A lot 

236. Mow much encouragement was given to students in academic matters? 

** 1 = Teacher gave very little or no encouragement to students; they were 
expected to assimilate the material on their own; also they were all 
expected to be self-motivated. 
5 = Teacher gave much encouragement to students, pushed students per- 
haps to their limits; constantly was encouraging them in their aca- 
demic pursuits. 

237. What was the negative affect rating for ^the classroom during a t>T)ical 
** observation? (Coldness, negativity, criticism, hostility from anyone.) 

1 = Several moderate or 1 or 2 more severe negative instances 

2 = Several mild or 1 or 2 moderately negative instances 

3 = 2 or 3 mildly negative instances 

4 = One mildly negative instance 

5 = Completely neutral 

238. What was the positive affect rating for the classroom during a typical 
** observation? (Positively supporting or reinforcing teacher attitudes or 

behavior.) 

1 = Completely neutral 

2 = Some positive affect, but perfunctory 

3 = Occasional, but low key, positive affect*' 

4 = One to two obviously sincere reinf orccme?\ts 

5 = Three or more genuinely supporting instances 

239. What kind of teacher clarity was observed (observer's ability to under- 
** stand what teacher is saying or doing)? 

1 = Very low 
5 = Very high 



1r , ^ 
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2^0- How much emphasis did teacher place on form? 

1 = Heavy stress, perhaps even over content. Teacher demanded that 
things be said and done in a very specific way; did not accept 
cc rect responses if the form is not correct. 
= Little or no concern about form of responses or assi ^^^nmcnt s . Teachei 
exhibited most concern over connent of res[)onses. 



n wliat ways did students receive feedback from the teacl^er? (Check all 
) Notes on papers that apply.) 

) Messages in small groups 
) Grades 

) Papers on bulletin board 
) Verbal citing in front of class 
) IndiA/'idual conferences with teacher 
) Qther, specify: 

248. Did teacher consistently plan enough work for students during a typical 
** observation? 
1 = Never 
5 = Always 



241. 
242. 
245. 
244. 
245. 
246. 
247. 



249. Were typical assignments too short or easy? 
** 1 - Never 

5 = Always 

250. Were typical assignments boring, repetitive or monotonous? 
1 = Never 

5 = Always 



251. Were assignments too hard; students couldn't get started, or continually 
needed help? 

1 = Never 
5 = Always 

252. How often would the teacher allow an activity to continue too long, 
* until it got boring? 

1 = Never 
5 = Always 

teacher assign appropriate assignments? 
1 = Never 
5 = Always 

254. How often did the teacher fill time with off-the-cuff activities? 
1 = Never 
5 = Frequently 

25.^. How sensitive was the teacher to students* attention span in pacino lessons 
1 = Not at all . * ^ . ^ 

5 = Very 

256. How often did teacher assign homework? 
1 - Never 

5 = Frequently ^ 

257. How often did teacher assign homework as a punishment for classroom 
misbehavior? 

1 = Never 

5 = All the tiine 1^3 

TIT-SS . . 
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2()(). 
261 . 
262. 

263. 

264. 



258. Mow often would the teacher assign homwork if work was not finished during 
class? 
1 = Never 
5 = Always 

I'or whom would teacher assign homework? 
( ) Only for students having problems 
( ) Only for those who didn^t finish classwork 
( ) Only for those who were misbehaving 
( ) Combination of above 
( ) For everybody 

UTiat was the efficiency of transitions? 
1 = Usually had overly long transitions 
5 = Mostly smooth, efficient transitions 

What did the teacher coiimionly do with children classified as having a learning 
problem of some kind? (Check all that apply.) 

265. ( ) Cave extra help to student 

266. ( ) Sent the student to a resource room; did nothing else 

267. ( ) Cave special materials to the student 

268. ( ) Formulated some kind of plan to put in use 

269. ( ) Some combination of the above 

270. ( ) Other, specify: 

271. How many students with special needs were present in the classroom? (either 
formally identified or those who haven't been, but probably could be.) 

1 = 1 . 

2 = 2 

3 = 3 
4=4- 

5 = 5 or more 



272. 
275. 
274. 
275. 
276. 
277. 



278. 
279. 
280. 

281 . 



What categories could these students be placed in? (Check all that apply.) 

(') Cifted 

( ) LD or slow learner 

( ) Behavioral ly disordered 

( ) Physically handicapped 

( ) Hard~of-hearing 

( ) Visually handicapped 

liow do you know this information? 
( ) Overheard teacher discussing it 
( ) Can observe it 
( ) Other source, specify: 

How accepting did this teacher appear of these students? 
1 = Not at all ' \ 

5 - Very 



282. 

* * 



How affectively integrated was/were these students? 
1 = Not at all 

5 - Very [ j 



ERIC 
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283. How were students with behavioral disturbances handled? 
** 1 = Very poorly; teacher's attention probably aggravated the situation 
5 = Well; with a minimum amount of embarrassment to student 



284. How much student obedience to the teacher was displayed? 
** 1 = Students commonly defied teacher and were disobedient. 

3 = Students were neither always compliant nor always disobedient; some 

were disobedient some of the time, but others were not. 
5 = Students were almost always compliant and obedient. 

285. How much did teacher socialize with students? 
1 = None or almost none. 

3 = Some small amount but usually before or after class 

5 = Teacher socialized with kids quite a bit, may even take place during 
class; teacher seemed very concerned with socializing with students. 



286. What was teacher's degree of displayed patience in correcting errors? 
** 1 = Teacher was impatient; embarrassed students, 

5 = Teacher was patient and supportive in dealing with tie student after 
he has made an error. 

287. What was the overall confidence level of the teacher? 

** 1 = Uncertain and lacking any overt confidence in what s/he was doin^^. 
5 = Very confident and assured as a teacher. 



288. How at ease was teacher in working with other adults present in the room? 
r = Not at all 
5 = Very 



* * 



289. 



How frequently was there another adult in the classroom (excluding the observer)? 
1 = Not at all 
5 = Very 



290. With another adult in the room, how comfortable did teacher appear in dele- 
gating authority? 

1 = Not at all 

5 = Very - ' 

291. How enthusiastic was this teacher? 

** 1 = Very unenthusiastic , didn^t appear to like teaching at all. 

5 = Very enthusiastic; obviously enjoyed job; conveyed it to students. 

292. Kliat kind of sho^anship (showomanship) did this teacher display? 



1 = Teacher was even-spoken, non-dramatic (although teacher may have been 

enthusiastic in non-dramatic w^ys) . 
5 = Teacher was melodramatic, expressive, gushy, "show-offy," etc. 

293.r Did teacher make productive use of own mistakes? 
1 = No, tried to deny or cover up mistakes. 

3 = Did notice them, or corrected th^m quicklyMvitliout calling any attention 
to them. 

5 = Called attention to mistakes, laughed at self or used the occasion for 
teaching or motivating students. 
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294. Did students seem eager to respond to teacher with no observed fear? 

** 1 = Students seemed resistent or fearful; didn^t raise their hands unless 
they were sur^. 

5 = Students blurted out answers, waved thoi^ -irms, seemed very eager to 
respond to teacher's questions. 

295. liow were ^classroom interruptions that wt . v the teacher's control 
** handled? 

1 Not well at all; very much interrupts tli*- flow of thiny.s 
5 = Very well; teacher kept things going 

29b. How apparent were problems with the machines of the room (record player, 
listing station, overhead projector)? 
1 = Not apparent 
5 = Very apparent 

297. How apparent were problems with consumable supplies (paper, paints, etc)? 
1 =Not apparent 
5 =Very apparent 

29S. How apparent were problems with textbooks? 
1 =Not very 
5 =Very apparent 

299. How supportive would you say this entire class (both \ teacher and students) 
is of each other? 

1 *^ Not supportive at all; every kid for himself. 
5 = Very supportive. 

300. How much of an awareness of the coder was evident in the teacher? 
** 1 = Not much at all 

5 = A lot 

301. How much of an awareness of the coder was evident in the students? 
1 = Not much at all 

5 = A lot 

302. How true is this statement, "This teacher likes kids." 
** 1 = Not very 

5 = Very 

303. How true is this statement, "The kids in this classroom really liked their 
** teacher." 

1 = Not very 
5 = Very 

504. During a typical observation, how often did the teacher spend time at liis/her 
** desk (actually sitting down and doing something there)? 

1 = Never 

2 = Maybe once 

3 = Twice 

4 = Three times, or for an extended period of time 

5 = At every opportunity 

30S. During a typical transition from one activity to anotlver, what was the 

** usual noise level? i 4 t / 

1 = Low ' ^ 
5 = High 
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306. How long did the typical transition last? 
** 1 = 1 minute or less 
2=1-3 minutes 
3=3-5 minutes 
4..= 5-7 minutes 
5 = More than 7 minutes 




S ~ Very closely 
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SECTION IV 
DATA ANALYSES 



Many data analyses were carried out by each R&D program. The 
PAEI/CBAM Program scored the LoU interviews and the SoC questionnaires 
so that profiles of all teachers would be available for these two 
measures. The EOT Program was involved in securing CAT scores for 
students in the observed classes. The COET staff performed both 
quantitative and qualitative analyses on the data sets in the COS. 
These analyses will be briefly summarized in this section. 
Preliminary Analyses 

1. Observer agreement on the Component Ratings was estimated in 
several ways, including intraclass correlations between observer pairs 
on each scale during the first three weeks* and during the 
remainder-of-year observations. In addition, correlations between the 
average ratings from the two time periods were calculated. Ratings were 
retained for subsequent analysis only if they showed adequate observer 
agreement, that is, a statistically significant intraclass correlation 
(p £ .05) or, in the case of marginal agreement, if the correlation 
between average ratings during the twd time periods was significant 
(p £ .05). The latter case indicated stable measurement of the rated 
characteristic over time. Of the original 32 variables, 25 were 
retained. These reliable variables are listed in Table 1. 

2- Student Engagement Ratings were obtained every 15 minutes 
during each observation. A frequency tally; was ma'de of the number of 
students engaged in academic, procedural, or off-task activities. Each 
variable was expressed as the percent of students who were classified in 
each^category. To check reliability the SERs were listed sequentially 
and separated into two sets, in odd-even fashion. Each SER variable was 
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then estimated from each set by averaging the remainder-of-year 
observations. Reliabilities were estimated' by correlating these 
split-half averages and applying the Spearman-Brown prophecy formula to 
estimate the reliability of the combined sets of observations. The 
estimated reliability of the engagement rate in all content activities 
was .85; in reading/language arts activities only, the reliability was 
.86. The reliability of on-task, academic activities was .71 for all 
content areas, and .76 for reading/ language arts alone. The reliability 
of off-task (unsanctioned) behavior was .91 for all content, and .89 for 
reading/ language arts alone. Thus the *?ngagement rates were highly 
reliable. 

In addition, the stability of the engagement measures across the 
school year was estimated by correlating the average percentages of 
students in various engagement categories from the first three weeks of 
the school year with those from the remainder of the year. The 
relationship between engagement rates for all activities was .51; for 
only academic activities, it was .46; and for off-task behavior 
(unsanctioned), the relationship was .54. ^ 

3. Intraclass correlations for the 307 Observer Ratings of 
Teachers indicated that only 96 of these variables were reliable . 
However, many reliabilities could not be computed because of low 
frequencies of occurrence or because there was no variance. The 307 
observer rating variables are listed on page and the 96 reliable 

variables are denoted by asterisks. These 96 itenv^ were factor analyzed 
using a principle components analysis with varimax rotation and 14 
factors resulted. Of these 14, only three were readily interpretable . 

1 1 0 
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They were named as follows: General Management, Teacher Relationships 
with Students, and Classroom Constraints. 

Factor scores for each teacher weref created for the above three 
factors using unit weighting of items. The 17 items loading highest on 
the General Management factor (loadings greater than .82) were chosen to 
compose that factor. The 18 items loading highest on the Teacher 
Relationships with Students factor (loadings greater than .69 were 
chosed to compose that factor. Finally, items with loadings greater 
than .58 were chosen to compose the Classroom Constraints factor. 

Teachers' scores on these factors were used as criteria to which 
other process measures were compared. Each factor with its 

0 

corresponding loading is listed in Table 2. 

4. The narrative descriptions were condensed by three readers, who 
prepared narrative summaries focused on several areas: behavior 
control, instructional management, meeting student concerns, physical 
arrangements, constraints on the teacher, and personal char acterist ics - 
The readers worked together -on several summaries until agreement was 
reached on the nature of the information to be included in the 
summaries. Narrative records were then divided into two sets: one set 
for the first three weeks and another set for the remainder-of-year . No 
reader was responsible for both the initial summary and the 
remainder-of^year summary for a teacher. 

5. In addition to preparing a summary. Reader Ratings were 
obtained by having each reader assess the teacher'^ adequacy in each of 
the five areas: controlling behavior , instructional management , meeting 
student concerns, physical arrangement, and constraints. The latter was 
given a dual rating: the severity of the constraints faced by the 
teacher and the degree to which the teacher was able to cope with them. 
A sixth rating was created by combining scores on the first two ratings, 



behavior control and instructional leadership, as these were considered 
the most important ratings regarding classroom management. Therefore, 
this sixth rating was expected to be a more general measure of the 
teacher's effectiveness in the classroom. Initial checks among the 
readers indicated that these characteristics could be reliably rated 
from the narrative records. This provided some assurance that the 
narratives could be used to characterize the organizational and the 
management behaviors of the teachers. The stability estimates of these 
ratings are shown below: 

Correlation of Reader Ratings 

Beginning- versus End-of-year 
(n = 27) 

Variab le . Correlat ion 

1. Behavior Control .83 

2. Instructional leadership .74 

3. Student concerns .68 

4. Room arrangement . .41 

5. Constraints .61 
Coping with constraints .76 

6. General effectiveness .86 



An indication of the stability of the narrative summary ratings over the 
school year was obtained by correlating beginning-jof -year ratings and 
remainder-of-year ratings. These correlations are reported on 
page IV- 14. All correlat ions were significant (p ^ .05) and usually 
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RiOderate to high. It should be noted however, that the narrative 
Variables _1^^ 2^. and 3_ are highly intercorrelated. These 
intercorrelat ions may be the result of actu^^sl inC.e'ri'',ependence ; for 
example, good behavioral managers may also be good instructional 
managers. These intercorrelations may also indicate halo on the part of 
the raters. Thus, although we will continue to distinguish conceptually 
between these management domains, they are undoubtedly linked. From the 
stability analysis, it is reasonable to conclude that the management 
characteristics of the teacher during the school year are at least 
moderately, and in some areas, highly predictable from the behavior of 
the first three weeks. 
Relationships Between Variables 

All of the data subsets previously mentioned were used as 
indicators of teachers' managerial skills. It is also informative to 
note how these measures relate to one another and to adjusted 
achievement as measured by the California Achievement Tests. The 
following matrices in Tables 3 and 4 show these relationships. The 
first matrix contains variab les for the first three weeks of observation 
and the second matrix contains variables from the remainder of the 
year. 

Selection of More and Less Effective Teachers 

At this stage of the data analyses , after running re liabi li ty and 
stability checks, and investigating the interrelationships between 
variables, we felt that the data were suf f i6iently^ dependab le to attempt 
to describe the teacher's management characteristics. For that purpose, 
subsamples of contrasting teachers were selected. Teachers who had 
initially comparable classes, but who differed in teaching effectiveness 
were selected using ?\ number of criteria. Using the California 
Achievement Test reading score obtained by each student . in April of 
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1977, class mean CAT scores were computed. Classes were , ranked and 
divided into three groups so that the subsamples were balanced according 
to CAT level. Teachers were ranked on management criteria taken from 
end-of-year student engagement ratings, reader ratings (done 
by narrative readers), and observer ratings of teachers. Class mean 
residual gain scores on CAT reading were computed using April 1978 
scores regressed on the previous year's scores. These were used a§ a 
check to be certain more and less effective managers were ^ different iated 
with respect to reading achievement. 

The subsample selected consisted of seven more and seven less 
effective teachers. The two groups were distributed evenly across 
schools, including Title I and non-Title I schools. 
Beginning of the Year Comparisons Between More and Less Effective 
Teachers 

The beginning-of-year activities of the managers designated as more 
effective and less effective were compared statistically by _t^-tests of 
the narrative ratings of management areas, the student engagement and 
off-task variables; and the instrlictional component r&txngs... These 
results are shown in Table 1. Many of these contrasts were 
statistically significant, indicating clear differences in management 
effectiveness at the beginning-of-year. In addition to the statistical 
treatment of. the data, the narrative records from the first three weeks 
were analyzed according to each of the narrative management areas, in 
order to describe the activities and behaviors of ^he two groups of 
teachers. Many differences were apparent and are summarized briefly 
below, 

1 

1. M \j 
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1. The more effective organizers had thought in advance of the 
rules and procedures necessary in their classrooms, and saw that these 
were established and communicated to the students before problems 
arose. 

2. The more effective organizers considered that the teaching of 
these rules and procedures was a very important part of instruction 
those first few weeks. That is, they taught "going to school" skills 
by providing practice in moving through procedures, responding to 
signals, etc., and pointed out to the students when they were behaving 
appropriately. They spent as much time as was necessary to establish 
basic routines and smoothly running procedures. 

3. The more effective organizers were able to "see through their 
students' eyes" in planning the classroom and introducing the students 
to the school year. That is, they could predict what would confuse or 
distract a student, and what would be of immediate concern to him or 
her. They used this information in such a way that problems were 
prevented. 

4. The more effective organizers introduced their students to 
independent work gradually, and did not "let go" of them without close 
supervision until they had communicated certain. expectations to the ' 
students and had established credibility as the leader of the 
classroom, 

* 

For other information sbout the findings from this study, please 
consult the following publisbed articles: 

Emmer, K. , Evertson, C. , & Anderson, L. Effective classroom . 

management at the beginning of the school year. Elementary School 
Journal , 1980, 8£, 219-231. : — ' 
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Evertson, C. , & Anderson, L. Beginning school. Educational 
Horizons , 1979, ST^, 16A-168. 

Anderson, L., Evertson, C. , & Ernmer, E. Diniensions in classroom 
^ management derived from recent research. Journal of Curriculum 
Studies, in press. 
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Table 1 

Component Ratings, Student Engagement Rates, and Reader Ratings 
for More and Less Effect ive Managers 
during the First Three Weeks 

More Less 

Effective Effective 

(n =7) (n = 7) t_ 

Variable x SD x SD (df = 12) ^< 



Behavior Management 



1. Variety of rewards 


A 


.3 


1.03 


3 


.1 


M 


2 


.A8 


.05 


2. Signals appropriate 




















behavior 


5 


.A 


.70 


3 


.8 


.87 


3 


.81 


.01 


3. Eye contact 


6 


.1 


.A3 


A 


.9 


.81 


3 


.5A 


.01 


4. States desired 




















attitudes 


5 


.5 


.79 


3 


.9 


.81 


3, 


.77 


^ 01 


5. Reinforces inattentive 




















behavior 


2 


. 7 


1 .29 


3 


.6 


1.50 


-1 , 


.19 


ns 


6. Disruptive pupil 




















behavior 


3 


.0 


1.31 


A, 


.8 


1.06 


-2, 


.82 


.05 


Pesponses to Disruptive 




















Behav ior 




















7. Stops quickly 


A, 


.9 


.70 


3, 


.5 


1.03 


3. 


.10 


.01 


8. Criticizes 


1 


.8 


.79 


2 


.A 


1.16 


-1, 


.16 


ns 


9. Punishes (frequency) 


1, 


.6 


.A8 


2, 


.0 


.85 


-1. 


2A 


ns 


10. Ignores 


2, 


.9 


.77 


3, 


.6 


.55 


-1. 


,95 


.10 


11. Time out (frequency) 




.9 


.35 


1 . 


.0 


.79 




35 


rs 


Instructional Management 




















12. Describes objectives 




















clearly 


5. 


,1 


.72 


3. 


,1 


.97 


a'. 


AO 


.01 
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Table 1-Continued 

More Less 
Effective Effective 



Variable 


(n = 

X 


7) 
SD 


(n = 

X 


7) 
SD 


t 

(df = 12) 




13. Uses a variety of 
materials 


5.6 


.70 


3.7 


.58 


5.66 


.01 


14. Materials are ready 


'6.2 


.56 


4.4 


.84 


4.62 


.01 


15. Materials support 
^ instruction 


6.0 


.57 


4.3 


1.07 


3.55 


.01 


16. Clear directions 


5.2 


.80 


3.8 


.84 


3.14 


.01 


,17. Clear presentation 


5.8 


.59 


4.1 . 


1.26 


3.22 


.01 


18. Provides/seeks 
rationale or 
analysis 


■4.9 


1.07 


3.4 


1.13 


2.48 


.05 


Meeting Student Concerns 














19. Attention spans 

considered 

in lesson design 

20. High degree of pupil 

success 


5.2 

,5.5 


4.70 
.63 


2.8 
3.9 


.78 
.72 


6.23 
4.22 


.01 
.01 


. 21. Content related to 
pupil inteirests 


5.2 


.61 


3.6 


.45 


5.99 


.01 


22. Reasonable work 
standards 


5.8 


.47 


4.6 


1.03 


2.83 


.05 


Personal Characteristics 














,23. Distracting mannerisms 


1.9 


.52 


1.6 


.32 


.99 


ns 


24, Listening skills 


5.4 


.76 


3.8 


.67 


4.30 


.01 


25. Expresses feelings 


5.0 


1.08 


3.2 


.22 


4.41 • 


.01 



Student Engagement Rates 

26. On-task, all activities . .86 .06 .75 .09 2.72 .05 
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Table 1-Continued 





Variable 


More 
Effective 
(n = 7) 
X SD 


Less 
Effective 
(n = 7) 
X SD 


Idf - 12) 


S. < 


27. 


On-task, in content- 
(not procedures) 


-65 


.06 


.59 


.06 


1.92 


.10 


28. 


Off-task, unsanctioned 


.07 


.03 


.16 


.09 


-2.57 


.05 


Reader Ratings 














29. 


Behavior control 


4.43 


.53 


1.43 


.79 


8.36 


.01 


30. 


Instructional 
leadership 


4.14 


.69 


1.57 


.79 


6.50 


.01 


31. 


Student concerns 


4.29 


.76 


1.71 


.76 


6.36 


.01 


32. 


Room arrangement 


4.29 


.76 


2.57 


.98 


3.67 


.01 


33. 


Constraints 


3.29 


.49 


3.43 1 


.51 


.24 


ns 


34. 


Coping with 
constraints 


4.57 


.79 


2.00 


.82 


6.00 


.01 


35. 


General effectiveness 


8.57 1 


.13 


3.00 1 


.53 


1.15 


.01 



Note. Except for Variables 9 and 11, which are frequency counts 
per observation, the scores for Variables 1 through 25 are average 
ratings made during the beginning-of-year observations. The scales 
range from 1 to 7, where a 1 represents little or no evidence of the 
rated characteristics or behavior and a 7 indicates relatively high 
amounts or frequent occurrences. Variables 26 through 28 are average 
proportions, based on frequency counts at 15-minute intervals. 
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Table 2 
* Observer Ratings Factors 
Loading Description 



General Management 
-.95 Noise level during typical transition 

-.94 Level of student talk during individual activities 

.94 Frequency of non-task related wandering 

.94 Student obedience 

.93 Degree of orderliness in lining up to leave room 

.91 ' Student self-control in remaining on-task during typical 
observation 

.91 Efficiency of transitions 

.91 Time class begins to lose concentration in morning 

.91 General noise level of class 

.91 Amount of socializing among students when working 

.88 Self-directedness of students entering room, readying for 

school 

-.87 Frequency of class getting beyond teacher control 

.86 Consistency of adequate work for students 

-.86 Length of typical transition 

.85 Consistency of success of attention-getting signals 

.84 Room runs smoothly, routines are automatic 

.82. Extent of acting up when/teacher leaves room 

Teacher's Relationship with Students 

.93 Teacher's leadership style is democratic 

\ 

.85 Teacher shows patience in elaborating instructions 

.86 Teacher accepting/integrating student ideas into class 

discussion 
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Loading 



Table 2-continued 
Description 



Teacher's Relationship with Students 



-.81 
.77 
.75 
.75 

75 
.74 
.74 
.7^ 
-.73 
.71 
.71 
.71 

.92 
-.90 

-.58 
.58 



Teacher seemed to like the students 

Degree of patience displayed by teacher in correcting 
errors 

Students seemed to like the teacher 
Teacher used as punishment: scolding 
Students' eagerness to respond without fear 
Teacher's acceptance of students with special needs 
Support iveness of teacher and studenr.s of each other 
Response to unsanctioned b^ihavior: order to stop 
Affective integration of students with special needs 
Positive alrfect 

Effect ivenass of hsndli'-.-'^ behavioral disturbances 
Frequency of loud correcc ions by teacher 
Teacher enthusiasm 

Ai^ount of . encouragement in acad<^^uic areas 

NegaL ive af feet 

Cl assroom Cons t 7;e?nts Fact o r 

Crowdeciness of classroorr. with most efficient use of space 

Crowde.dness of classroom, number nf students versus 
dimensions of room 

Comfort level of room through year 

Responsv to m; ^nctioned behavior: ^isolation or 

S2Dav.^t ion 

Adequacy of space for students* belongings 




Off-task, UnsancTioned 

On-Task Engagement 

General Effectiveness 
Reader Rating 

Reading achievement 

Math achievement 

Reading resicjual 

Math residual 

Observe-r Rating 
Factor Score 



Table 3 

Correlation Matrix of Management and Achievement 
Variables From the First Three Weeks of School 

(n = 26) . 



Off-task, On-task 
Unsanc- Engage- 
t ioned ment 

- . 92** 



General 
Effec- 
tiveness 
Reader 
Rating 

-.71** 

.77** 



Reading 
achieve- 
ment 

-.27 

.35 

.24 



Math 
ach ieve- 
ment 

-.36 

.32 . 

.32 
. .81** 



Reading 
res idual 

.03 

.02 

.13 

.36 
.19 



Math 
res idual 

-.09 

.02 

.22 
.43* 
.47** 
.47** 



Observer 
Rating 
Factor 
Score 

-.34 

.41* 

.91** 

.04 

.09 

.34 

.20 



*-p < . 05, 
< .01. 



Table 4 

Correlation Matrix of Mana^,ement and Achievement 



Math residual 

Observer Rating Factor 
Score 





Variables From 


Lhe Rest of 
V n — ZD / 


the School 


Year 








• 


Off-task, On-task 

lino Q n o "~ T4 n f» a ^ 

tioned ment 


np np T*fl 1 
Effec- 
tiveness 

Rating 


Reading 
raenC 


Math 

ii Vi 1 1 r r> ~ 
clL. 11 i. C V c 

ment 


T? 0 a rl 1 n 0 ■ 
ixcciu J. 11^ , 

res idual 


l la Lll 

res idual 


Observer 
Rating 

13* iir» t* fsT 
r aU L w 1, 

Score 


Off-task, Unsanctioned 


-.88** 


-.89** 


-.20 


-.32 


-.12 


-.20 


-.83** 


On-Task Engagement 




.82** 


.15 


.29 


.18 


.11 


. .76** 


General Effectiveness 
Reader Rating 






.19 


.24 


.36 


.35 


.92** 


Reading achievement 








.81** 


.36 . 


.A3* 


.04 


Math achievement 










.19 . 


.47** 


.10 \ 


Reading residual 












.47** 


.34 



,20 



*£ < .05. 
**p < .01. 
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